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Chapter 1 
Introduction 

Betheny Gross and Jonathan A. Supovitz 



State Accountability 
Policy and Our Special 
Focus on High Schools 

American public education faces 
increasing pressure to demonstrate the 
competence of all of its students as they 
progress through the grades and, especially, 
as students exit their high schools. In 
response, policymakers are developing 
sophisticated accountability and support 
systems in efforts to steer schools toward 
improved performance. These systems, as 
illustrated in Figure 1, combine a set of 
academic goals and standards with a battery 
of incentives to focus and motivate 
organizational and curricular change. In 
addition, these systems often provide 
resources to support local reform efforts. 
Although accountability systems such as 
these are not new to the educational policy 
environment, the reauthorization of the 
Elementary and Secondary Education Act in 
January, 2002, ensures that accountability 
systems focused on academic outcomes will 
continue for some time to come. This report 
shines a spotlight on high schools, which 
bring students to the last benchmarks in the 
K-12 system, and analyzes the response of 
teachers, school administrators, and the 
district administrators to these policies. 

The importance of accountability in 
local, state and national policy over the past 
15 years has led researchers to examine the 
impact of accountability on student 
achievement in a variety of contexts. 



Although not all studies prove gains in 
student performance due to accountability 
(Klein, Hamilton, McCaffrey, & Stecher, 
2000), many studies suggest that these 
policies can and often do have an impact on 
the performance of students. For example, a 
RAND study analyzed National Assessment 
of Educational Progress (NAEP) gains of 
fourth and eighth graders attributed the 
performance gains to the states’ high-stakes 
accountability systems (Grissmer, Flanagan, 
Kawata, & Williamson, 2000). In another 
example, a recent Consortium for Policy 
The importance of accountability in 
local, state and national policy over the past 
15 years has led researchers to examine the 
impact of accountability on student 
achievement in a variety of contexts. 
Although not all studies prove gains in 
student performance due to accountability 
(Klein, Hamilton, McCaffrey, & Stecher, 
2000), many studies suggest that these 
policies can and often do have an impact on 
the performance of students. For example, a 
RAND study analyzed National Assessment 
of Educational Progress (NAEP) gains of 
fourth and eighth graders attributed the 
performance gains to the states’ high-stakes 
accountability systems (Grissmer, Flanagan, 
Kawata, & Williamson, 2000). In another 
example, a recent Consortium for Policy 
Research in Education (CPRE) study by 
Carnoy and Loeb ( 2004) also showed that 
high-stakes accountability has had a positive 
impact on student performance at the fourth 
and eighth grades over the years 1996 to 
2000. 
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Figure 1. Accountability Strategy foiy School Improvement 




These studies, and other studies of 
effects, can only infer that changes have 
been made to the educational program 
within these schools. Many policy 
researchers argue that we must pay careful 
attention to the instructional and 
organizational changes that occur with 
accountability, as some logical 
consequences of the policy such as teaching 
to the test, strategic targeting of students, 
cheating on assessments, and narrowing 
curriculum potentially compromise the 
benefits of the policy (Darling-Hammond & 
Ascher, 1991; Education Commission of the 
States, 1998; Elmore, Abelmann, & 
Fuhrman, 1996; Linn, 2000; McNeil, 2000). 
In this study we respond to these concerns 
by looking inside schools to see how 
accountability shaped the goals and 
improvement efforts described in high 
schools. 

In this study we also give attention to 
the high school organization which, due to 
the age of the students served and to 
organizational complexity, is often viewed 
as more challenging to study than the 
elementary school. High schools confront 



the challenge of working with adolescent 
students who express a great deal of agency 
in their schooling, which makes it difficult 
to separate the effect of students from the 
effect of their teachers or school. In 
addition, high schools tend to be larger 
organizations with many complex layers due 
to the specialized content focus and deeply 
held sense of professional autonomy held by 
high school teachers, making high schools 
very difficult to understand as a single 
organization. For these and other reasons, 
high schools have received less attention in 
the research community. However, high 
schools are of particular interest today in 
light of the emphasis accountability places 
on benchmarks and exit exams. 

This report by the Consortium for 
Policy Research in Education (CPRE) 
focuses squarely on strategies for 
instructional improvement in American high 
schools. Specifically, this study examines 
how high schools that perform below 
average incorporate their state’s 
accountability goals into their own goals, 
identify their challenges, and search for 
strategies for instructional improvement. We 
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focus on how high schools of differing 
performance levels and contexts, residing in 
states with different forms of high-stakes 
accountability and support systems, identify, 
understand, and respond to the gap between 
their current levels of performance and 
external expectations for their performance. 

Interpreting and 
Implementing: A 
Conceptual 
Framework of 
Accountability and 
School Context 

The theory of outcomes accountability 
and the assumptions embedded within 
accountability policies, like many state 
administered policies, faces considerable 
challenges in practice. The reality that 
carries these policies from paper into the 
schools is a complicated picture in which the 
state’s message is interpreted through local 
values and expectations (McLaughlin, 

1987). The school’s response is then shaped 
by local interests and constrained by the 
school’s ability to make change (Newmann, 
King, & Rigdon, 1997). Finally, as happens 
often in policy implementation, we can 
expect teachers and students to accept, 
challenge, and/or alter programmatic 
changes on the basis of their own 
expectation and motivation (Weatherley & 
Lipsky, 1977). The chapters in this report 
and the discussion of the framework, 
diagramed in Figure 2, of school response to 
accountability policy focus on the first two 
stages of policy implementation in which 
schools interpret the state policy and 
become motivated (or not motivated) to 
focus on the policy’s goals and standards 
and when they focus their resources and 



other support to generate a response to the 
policy. These chapters do not attempt to 
evaluate the impact of accountability on 
student outcomes. Instead we discuss the 
implementation of state accountability 
policies and the impact of these policies on 
the goals and activities of teachers, school 
administrators, and district administrators. 
As Hargrove ( 1983) explains, 
implementation is the extent to which the 
target of the policy “goes beyond 
compliance to incorporate the required 
action into the organizational routines of the 
implementing agencies” (p. 281). 



3 




Figure 2. Conceptual Framework of School Response to Accountability Policy 



State policy is interpreted by local 
actors (school and district level) 



School Response: Search, decision 
making, and strategy selection 




Districts’ support for change in high schools 
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The Acceptance and 
Interpretation of the 
Policy 

The acceptance of the policy and 
appropriate interpretation of the policy 
by those at the school and district level is 
an important condition for the 
implementation of the policy at the local 
level. However, obtaining this condition 
is not a simple matter of announcing the 
goals and articulating sanctions. This 
process may take time and may not 
happen at all. Individuals in schools 
must hear and be compelled to 
acknowledge the policy and the 
expectations laid out in the policy at 
least amid, if not above, the chatter 
generated by the schools many 
stakeholders inside and outside the 
school. Teachers and administrators 
must then interpret, hopefully in concert, 
the policy and its components as 
intended by the state policy designers. 
While a great many issues influence the 
acceptance and interpretation of policies 
by policy targets, in the chapters that 
follow, the authors focus on two 
influences on policy interpretation: (a) 
elements of the policy design to make 
the policy clear, reliable, and stable and 
(b) the filtering of varied interests in the 
local context in interpreting the state’s 
intentions and accommodating local 
interests. 

Of the many aspects of the policy’s 
design, clarity and stability of policy are 
of particular importance to the 
acceptance and interpretation of state 
accountability policy. Researchers of 
policy and policy implementation have 
continuously argued that policy clarity is 
very important in that it reduces the 
likelihood that policies will be 



misinterpreted or that the policy targets 
will fail to implement the policy out of 
confusion (Baier, March, & Saetren, 
1988; Hargrove, 1983; McDonnell & 
Elmore, 1987; Odden, 1991). Given that 
focusing educators’ attention on specific 
performance targets and accountability 
standards is the centerpiece of this 
policy, clarity is of critical importance to 
accountability policies. Since the early 
days of outcomes accountability, states 
have had to negotiate the trade-off 
between varied and multiple assessment 
and measurement with the transparency 
of accountability. Many educators argue 
in favor of using multiple assessments 
that recognize multiple outcomes (e.g. 
achievement, discipline, and dropout) as 
well as accounting for growth and 
student subgroup performance. 

However, incorporating each of these 
elements into a performance measure 
leads to a complicated rubric for 
evaluating schools. Such a scaling 
technique that evaluates a set of 
outcomes and creates a composite score 
that makes statistical adjustments for 
reliability generates scores that are 
difficult for schools to predict on the 
basis of the performance reports they 
have at hand. In addition to clarity of 
performance goals, the policy must also 
demonstrate consistency and 
predictability in its incentive structure. 
Schools cannot be expected to respond 
to incentives if they do not know or 
understand the basis on which they will 
be delivered (Brooks, 2000). For 
example, we found that the performance 
bonuses offered to Pennsylvania schools 
were generally not a salient component 
of the accountability system. Few 
Pennsylvania teachers we spoke with 
commented on these rewards or 
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understood how they received these 
rewards. 

Researchers observing 
accountability policy argue that 
alignment to standards with performance 
targets presents a significant change for 
schools (McDonnell & Elmore, 1987), 
especially high schools, which have 
offered differentiated curriculum and 
whose teachers have exercised 
considerable autonomy for decades 
(Siskin, 2003). For schools to embark on 
the change expected by these policies, 
schools must be assured that the policy 
will last. The stability of policies and 
administrators’ commitment to policies 
over time give the policy targets time to 
learn about and understand the policy 
and thereby increase the likelihood that 
the policy targets will see value in 
responding to the policy and engaging in 
the goals of the policy (Hargrove, 1983). 
The six states in our sample have had 
their policies in place for different 
lengths of time and have had different 
degrees of the change over the years. In 
this report, researchers comment on how 
the maturity of the state policy seemed 
to impact the response of schools we 
visited. 

Beyond the elements of policy 
design, the goals and practices 
articulated by the school respondents 
reflected the influence of the local 
context including the values, beliefs, and 
expectations of local teachers and 
administrators as well as external 
expectations placed on schools. While 
the policy administrators may have 
control over the design of the policy and 
to some extent the stability of the policy 
over time, these policies enter local 
environments that vary across the region 
in which the policy is applied and that 
the policy administrators often have little 
control over (McLaughlin, 1987). These 



local environments are vitally important 
to the acceptance and interpretation of 
the policy. The motivation of policy 
targets to accept the policy and its goals 
depends on how well their own 
assessment of what should be 
accomplished aligns with the 
expectations embedded in the policy’s 
goals (Hatch, 1998; McLaughlin). 
Teachers who do not believe that the 
state assessment accurately reflects their 
students’ skills or who do not think the 
standards reflect appropriate material for 
their students will be less motivated to 
adjust their lessons or curriculum to 
align with the assessment or standards. 
Even though the incentive structures 
built into accountability policies intend 
to force local interests into alignment 
with the policy goals, McLaughlin 
argues that these incentive structures 
confront very powerful local norms and 
values that can pose a substantial 
challenge to the policy when they do no 
align. These situations require a strong 
and long-term commitment by the state 
to the policy. 

The local context, however, 
includes much more than the values and 
beliefs of individuals inside the school. 

A great number of educational 
stakeholders such as parents, local 
community members, business 
members, and educational organizations 
outside the school compete with each 
other and the states to influence local 
schools toward their agendas. Schools 
then filter these external interests 
through the values, expectations, and 
goals of teachers, administrators, and 
stakeholders. They also prioritize their 
goals and interests on the basis of the 
relative authority and influence of each 
of their stakeholders. Local agents 
position the goals embedded in the state 
accountability policy among the various 
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goals, according to the alignment of state 
the state board of education, and the 
influence gained through an incentive 
structure. The chapters that follow 
illustrate the competing interests present 
in the schools and the extent to which 
states’ accountability goals and 
expectations focused attention in 
schools. 

Generating a Response 

For accountability to impact the 
practice of teachers and the organization 
of schools, the policy must do more than 
focus teachers’ and administrators’ 
attention on the policy’s goals. People in 
schools must now respond by devoting 
their own resources, pulling in outside 
resources, drawing from their districts, 
and utilizing any assistance offered 
through the state policy to seek, select, 
and implement changes in the school. 
The ability to generate a change 
response by organizations as well as the 
nature of change in schools is central to 
understanding the impact of 
accountability in schools. Of particular 
concern is the extent to which the 
pressure and focus provided by the 
policy combined with the local resources 
and interests lead to instructional and 
organizational changes that hold the 
potential to create long-term 
improvement in schools. 

Unfortunately, a response that leads 
to long-term, consistent improvement is 
not automatic, even in cases in which the 
local teachers and administrators have 
acknowledged and incorporated the 
policy’s goals into their own goals. 
Actions taken (or not taken) by schools 
in response to accountability will be 
shaped by the interests of teachers and 
administrators, and perhaps more 
importantly, the changes pursued will be 



goals with local goals, the authority of 
shaped by the resources, knowledge, and 
skills (often referred to as capacity) 
available to schools attempting change. 
As Hargrove (1983) points out in a 
discussion of regulatory policy, the 
capacity of the policy targets is a critical 
issue determining policy 
implementation. The ability to make 
change is of particular importance in the 
context of accountability in high schools 
because, as mentioned before, these 
policies expect substantial change from 
many high schools but the policy rests 
on the notion that schools should be free 
to select and enact their own strategies 
for improvement. 

Although it may be argued that the 
sanctions should compel capacity 
development, many policy researchers 
and educational researchers question if 
spontaneous development of resources, 
knowledge, and skills is possible without 
substantial support from outside the 
organization. McLaughlin (1987) argues 
that pressure from policy alone does not 
necessarily imply a change in 
fundamental values and practice. For 
example, Newmann, King, and Rigdon 
(1997) found that prior capacity and 
capacity development in the school 
determined the extent to which a 
restructuring school improved under the 
context of high- stakes accountability. In 
another example, a study of high schools 
found that the internal coordination and 
coherence of the staff along with the 
alignment of these teachers’ beliefs with 
the policy’s goals predicted how well the 
schools’ response actually aligned with 
the intentions behind the accountability 
policy (Debray, Parson, & Avila, 2003). 

For these reasons, researchers 
observing and commenting on 
accountability have called on states 
enacting accountability to support 
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schools with programs and resources to 
& Elmore, 1999) or for the broader 
educational environment including 
districts and external providers to step in 
and provide support to schools in need 
of organization, curriculum, or 
instructional change (Brooks, 2000). As 
the authors of this report discuss the 
change and reform efforts seen in high 
schools, they discuss the support offered 
as part of the state accountability policy, 
the support offered by other external 
agents, particularly districts, and the 
resources and information both within 
and outside the school that were 
available for schools’ decision making. 

Methodology 

The results reported in this study are 
based on a nested sample of 48 schools 
in 36 districts in six states. Our sampling 
strategy and methodological approach 
was designed to confront a couple of 
important data collection issues. First, 
we wanted to focus on relatively low- 
performing schools, believing that these 
schools would be most affected by the 
state accountability policy. We also 
wanted a variety of school contexts to be 
represented in our sample. To meet these 
concerns we developed a sampling 
strategy — described in more detail 
below — that differentiated schools along 
both performance level and social 
context. Second, we wanted to include a 



assist in their change efforts (Abelmann 
relatively large sample of schools in 
states across the country, which 
introduced complications for data 
collection and analysis. To do so, we 
used techniques that would allow 
multiple researchers to visit schools, 
analyze transcript data for the schools 
they visit, and provide materials for a 
cross-case analysis. The details 
regarding our sampling strategy, data 
collection methods, and analysis 
methods are given below. 

Sample 

We started by identifying six states 
within which we planned to conduct our 
fieldwork. Using discussions of the 
strength of state accountability systems 
conducted by Goertz and Duffy (2001) 
and Camoy and Loeb (2004), we 
identified four strong accountability 
states and two weak accountability 
states, listed in Table 1. Our definition of 
a strong accountability state was one that 
had sanctions in place for schools and 
students during the 2002-2003 school 
year. A weak accountability state had no 
sanctions (but possibly rewards) at the 
local level for either schools or students. 
The state sample consisted of the 
following (a more detailed description of 
the accountability systems in these states 
can be found in Chapter 2 of this report). 
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Table 1 . Sample of Six State Accountability Systems 



State 


Student Accountability 


School Accountability 


California 


Strong in 2005 


Strong 


Florida 


Strong 


Strong 


North Carolina 


Strong 


Strong 


New Y ork 


Strong 


Strong 


Michigan 


Weak 


Weak 


Pennsylvania 


Weak 


Weak 



Within each of these six states, our 
sampling strategy was designed to 
produce eight schools with varying 
school performance and context. Our 
school selection process was deliberately 
developed to obtain a range in student 
achievement and context in order to 
examine the conditions under which a 
range of schools search for responses to 
their performance problem. 

To develop a sampling frame we 
collected school-level mathematics and 
English language arts achievement data 
and school context indicators for 1999— 
2000 for the population of high schools 
in each of the six states in our study. 
After observing a high correlation 
between the two subject-matter tests, we 
decided to use the mathematics 
assessment as the measure of school 
achievement, because this measure is 
often thought to be more sensitive to 
differences in schools’ instructional 
programs than is the measure of reading 
performance. In addition to math 
achievement scores, we collected all 
publicly available school context 
indicators including percentage of free 
and reduced-price lunch, percentage 
minority, percentage English language 
learners, and teachers’ years of 



experience. Because states collect and 
report different statistics on their 
schools, the number and nature of 
context indicators varied across the 
states in our study. 

Using only the below-average half 
of the sampling frame, we developed a 
regression model that predicted 2000 
school-level achievement, controlling for 
the available school context indicators in 
that state. We then produced a residual 
for each school to determine which 
schools performed better than, worse 
than, or as would be expected given their 
context. We examined the residuals for 
each school and placed each of the 
schools on a 9-cell matrix of predicted 
achievement relative to context reflected 
in Table 2. We focused this study on 
school in the highlighted cells because 
they represent a range of predicted 
performance and context. We selected 
eight schools that fit into the following 
categories: 

• one school that was 
underachieving given a 
relatively high context, 

• one school that was 
underachieving given an 
average context, 
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• one school that was 
performing as expected given 
a relatively low context, 

• one school that was 
overachieving given a 
relatively high context, 

• two schools that were 
overachieving given a 
relatively low context, and 

• two schools that were 
overachieving given an 
average context. 

As we selected schools for our sample in 
accordance with the rubric given above, 
we made efforts to select more than one 
school from some districts in order 
examine the differential impact of 
accountability in similar district 
contexts. We selected multiple schools 
from one district when two of the 
district’s schools fell into one of the six 
desired cells in the achievement/context 



matrix. Due to difficulties in obtaining 
permission from schools to visit, our 
sample included a handful of schools 
that fell on the borders of these 
categories. In addition, we only visited 
seven schools in New Y ork because of 
access issues. We visited nine schools in 
Michigan to accommodate district 
nesting. For the most part, however, our 
sample remained true to our intended 
sampling frame. 

The final sample consisted of 48 
schools nested within 34 districts 
representing a range of contexts. Figure 
3 shows that urban schools made up 
almost half of our sample, but rural and 
suburban schools were well represented 
in the sample. In addition, our sample 
included many different-sized schools as 
illustrated in Figure 4. Figures 5 and 6 
show that our sample included schools 
serving varied concentrations of free- or 
reduced-price lunch students and varied 
concentrations of ethnic groups. 



Table 2. Achievement/Context Matrix 



Under Predicted 
achievement 


Predicted 

Achievement 


Over Predicted 
Achievement 


Low Context Underachieving 


Expected 


Overachieving 


given a relatively 


achievement given a 


given a relatively 


low context 


relatively low 


low context 




context 


(2 schools per state) 




(1 school per state) 




Average Context Underachieving 


Expected 


Overachieving 


given a relatively 


achievement given 


given a relatively 


average context 


an average context 


average context 


(1 school per state) 




(2 schools per state) 


High Context Underachieving 


Expected 


Overachieving 


given a relatively 


achievement given a 


given a relatively 


high context 


relatively high 


high context 


(1 school per state) 


context 


(1 school per state) 
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Figure 3. Geographic Locale of Visited Schools 




Figure 4. Enrollment Numbers in Visited Schools 
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Figure 5. Percentage of Students Receiving Free or Reduced Lunch in 



Sample Schools 




Figure 6. Ethnic Composition of Visited Schools 




Predominately 

White, 

17 schools 



Predominately 

Hispanic, 

5 schools 






Mixed, 

19 schools 



Predominately 

African 

American, 

7 schools 
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Data Collection 

Fieldwork in each of the 48 high 
schools was carried out during the 2002- 
2003 school year and involved 
structured interviews with a set of school 
and district representatives. As with 
most studies of policy it is important to 
take note of the year in which the data 
was collected with respect to the policy 
context. The 2002-2003 school year is 
significant in that each of the states had 
implemented their current form of 
accountability prior to our visit, and all 
were just beginning to respond to the 
accountability provisions of the federal 
No Child Left Behind Act (NCLB). 

The structured interviews were 
carried out with district administrators 
(directors of secondary education, 
directors of assessment, curriculum 
specialists in English and mathematics, 
and the superintendent), school leaders 
(principals and assistant principals), 
department leaders in both English and 
mathematics, and English and 
mathematics teachers. We also 
interviewed the foreign language 
department chair to get a different 
perspective on the school and the 
perspective of a nontested subject. 
Because the question of who was 
searching for new instructional strategies 
was critical to our data collection efforts, 
we employed a sliding emphasis strategy 
in which the emphasis of our data 
collection efforts was adjusted to match 
our identification of the key individuals 
or groups who sought new strategies in 
that particular school. For example, 
although the core data collection always 
included interviews with district 
administrators, school leaders, 
departmental chairs and teachers, our 
protocols included auxiliary questions 
and probes to go deeper into the search 



process as we identified the locus of the 
search. These data collection efforts 
involved two researchers for two days in 
each school. 

Data Analysis 

On the basis of the framework 
developed for this phase of the study, 
using ATLAS .ti qualitative research 
software, researchers coded the 
interview transcripts using a coding 
system related to the interview protocols 
used in the field. The coding scheme was 
designed to highlight interview 
responses related to teachers’ goals and 
challenges, teachers’ understanding of 
accountability, the response of teachers 
to accountability, teachers’ perceptions 
of their school’s response to 
accountability, the process through 
which instructional and organizational 
changes were sought and implemented, 
and the nature of improvement strategies 
attempted in the school. The coding 
allowed the research teams to investigate 
patterns in the data they collected, and 
case reports facilitated cross-case 
analysis. 

Each of the research teams that 
visited a school used the coded 
transcripts to complete an internal case 
report for each of the schools and 
districts they visited. These case reports 
focused on the articulation of 
accountability, goals, challenges, 
searches for improvement strategies, and 
improvement strategies currently in the 
school. These case reports also described 
the patterns of response within schools 
and included extensive data extracted 
from the field transcripts. Researchers 
investigating the issues in this report — 
accountability press, decision making in 
schools, strategies employed by schools, 
and the response of districts — used the 
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data assembled into case reports to 
develop data matrices with focus areas 
relevant for their studies. These 
matrices, which compiled information 
from each of the 48 schools arranged by 
state, aided researchers as they looked 
for patterns across schools and states to 
inform cross-case discussions. 

Reading Across the 
Chapters 

The chapters that follow examine 
issues related to the interpretation of 
accountability policy and the response of 
high schools and districts. While the 
next four chapters draw from the same 
sources of data, each one brings a 
different perspective to these data. The 
chapter by Diane Mas sell, Margaret E. 
Goertz, Gayle Christensen, and Matthew 
Goldwasser and the chapter by Elliot 
Weinbaum discuss how school and 
district agents have interpreted the 
various components of their states’ 
accountability systems, while the chapter 
by Betheny Gross, Michael Kirst, Dana 
Holland, and Tom Luschei, that by 
Weinbaum, and especially that by Donna 
M. Harris, Melissa Prosky, Amy Bach, 
Karen Hussar, and Julian Vasquez Heilig 
highlight the local improvement 
response made by the high schools in 
this study. The chapters also differ in the 
level of the educational establishment 
they bring into focus. While Mas sell et 
al., Gross et al., and Harris et al. take a 
school-level perspective, Weinbaum 
discusses the interpretation and response 
of districts, which in many ways face a 
different set of incentives and sanctions 



from both the policy and their local 
contexts than their schools face. 

Chapters 2 through 5 are presented 
in an effort to show how the policy has 
unfolded for schools by beginning with a 
discussion of schools’ interpretation of 
the policy and continuing with 
discussions of the schools’ response to 
the press they experience. In chapter 
two, Massell et al. begin the 
conversation with their school-level 
analysis of the press teachers and 
administrators feel and attribute to their 
state’s accountability policy. This study 
of the press shows how school-level 
agents are interpreting and incorporating 
the goals and sanctions their states have 
put in place. Gross et al. follow in 
chapter 3 with a discussion of how local 
agents’ interpretation of their states’ 
policy shaped their prioritization of 
goals and challenges. This chapter also 
begins to discuss the nature of schools’ 
response to accountability by showing 
the influence of accountability on the 
decision-making process and a general 
picture of the range of strategies adopted 
by high schools in recent years. The 
work by Harris et al. in chapter 4 shows 
the types of strategies adopted by high 
schools in response to accountability 
pressure. The fifth chapter offers a 
different but very relevant perspective 
with Weinbaum’ s discussion of the 
accountability story as experienced by 
districts. We conclude the report with a 
discussion that looks across each of the 
papers to discuss the themes of states’ 
influence on local agents through 
accountability policy, the consequences 
of this influence, and policy directions 
states should consider as this policy is 
further developed. 
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Chapter 2 

The Press From Above, the Pull From Below: 
High School Responses to External Accountability 

Diane Massell, Margaret Goertz, Gayle Christensen, and Matthew Goldwasser 



Introduction 

In the research literature, high schools 
are often portrayed as the level of the 
educational system most resistant to reform 
initiatives. Unlike elementary schools, for 
example, high schools are balkanized into 
subject matter departments, teams, 
academies, and other substructures, making 
communication and influence very complex 
and challenging (see Siskin, 2004). Perhaps 
because of their reputation for intransigence, 
high schools have received comparatively 
less scrutiny and focused attention from 
policymakers than elementary and middle 
schools. 

Recent state standards-based reform 
initiatives, however, do include high schools 
under the umbrella of performance 
accountability. In contrast to conventional 
efforts to monitor school compliance with 
input and process regulations, these newer 
forms of accountability focus on student 
academic outcomes, schools’ continuous 
improvement on explicit performance 
targets, the public reporting of test results, 
and greater consequences for failure to 
succeed (see Fuhrman, 1999). Many states 
have also attached incentives for high school 
students to improve their performance on 
state tests, ranging from scholarships to 
grade promotion or graduation. 

This chapter explores high school 
teachers’ and school and district 
administrators’ response to their state 
accountability system. This system includes 
not just the particular design of 



accountability programs, but also the state 
tests and content standards to which they are 
tied. In the following section, we offer an 
overview of those systems for high schools 
during the time of our fieldwork in 2002- 
2003, the year preceding the implementation 
of the No Child Left Behind Act (NCLB). 
We evaluate the strength and stability of 
state accountability programs, and compare 
the nature of different state assessments and 
content standards. 

Then we look at whether educators 
were aware of and understood the 
accountability expectations, and explore 
their perceptions about its value. We 
anticipated that educators’ knowledge and 
awareness of the system and its demands 
(Abelmann & Elmore, 1999; Elmore, 
Abelmann, & Fuhrman, 1996) would 
contribute to a greater likelihood that they 
would act on its behalf. And ever since the 
early RAND studies of program 
implementation in the 1970s, it has become 
a truism that action on behalf of policies and 
programs is highly dependent upon the will 
and commitment of the “street-level 
bureaucrats” expected to carry them out 
(Berman & McLaughlin, 1975; Lipsky, 
1980). As expressed by one of the original 
authors of the RAND reports 15 years later: 

Policy cannot mandate what matters 
[italics added]. . . . The presence of will 
or motivation to embrace policy 
objectives or strategies is essential in the 
generation of the effort and energy 
necessary for a successful project. 
(McLaughlin, 1990, pp. 12-13) 
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This view is now deeply embedded in 
reform strategy. So, for example, when 
national and state education policymakers 
decided that content standards were an 
essential tool for school improvement, they 
tried to gain the consensus of the public and 
professionals through an inclusive standards 
development process. They assumed that 
such participation would produce the buy-in 
needed for strong implementation (Massell 
1994, 2000). 

Next, we discuss the salience of the 
accountability system to high school 
educators and administrators, e.g., the extent 
to which high school educators and 
administrators reported feeling pressure 
from the accountability system, and acted in 
response to its demands. The salience of the 
system predictably differed across schools. 
At the low end of the spectrum were schools 
where individual actors felt some pressure 
and took some initiative to address 
accountability demands, but the departments 
or school did not act as a whole. By contrast 
were schools that collectively launched one 
or sometimes many accountability-related 
initiatives. 

In the remainder of the chapter we 
analyze how these various factors — 
accountability system designs, knowledge 
and perceptions, and salience — and others 
that emerged as important (notably, district 
press and school leadership), contributed to 
high schools’ response to their state’s 
accountability system. 



Standards, Testing, 
and Accountability for 
High Schools in the Six 
States 

Accountability policies, as applied to 
high schools in 2002-2003 in the six states 
selected for study, varied on a number of 
dimensions that could affect educators’ 
understanding and acceptance of, and 
response to, state policies. These include the 
target of the accountability system (student, 
school, and/or district), the type of 
assessments used for student and 
institutional accountability, the nature of the 
accountability measure and its 
consequences, and the overall strength and 
stability of the accountability system. Table 
1 summarizes these variables across the six 
states, and Table 2 provides specifics about 
each state assessment; the tables appear at 
the end of the chapter. Note that we gathered 
information on state assessment and 
accountability policies from extant reports 
(Center on Education Policy, 2003; Goertz 
& Duffy, 2001), state Web sites and 
published information, and follow-up 
interviews with personnel from state 
departments of education. As described in 
greater detail later, these policies represent 
variation in the incentives used to capture 
schools’ attention, the assessments used to 
measure student performance, the support 
provided for school improvement, and the 
historical and political contexts of the 
policies. 

In this section, we look across state 
policies to discuss the strength of the 
accountability systems (as measured by the 
target and consequences of the policies) and 
their stability, the nature of the state testing 
program and its perceived alignment with 
state standards, and the specificity of state 
guidance. 
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Strength and Stability of 
Accountability Systems 

We categorized the six state 
accountability systems by the scope of their 
coverage (student, school, district) and the 
nature and strength of the consequences 
applied to the accountability targets. We 
consider four of the states — California, 
Florida, New York, and North Carolina — to 
have strong accountability systems, and the 
other two states — Michigan and 
Pennsylvania — to have weak accountability 
systems. 

Strong Systems 

The four strong systems each held high 
school students accountable through a high 
school exit examination. This requirement 
had been in place in Florida, New York, and 
North Carolina since at least the mid-1980s, 
although all three states have increased the 
rigor of their high school assessment over 
that period of time. Passing the Florida 
Comprehensive Assessment Test (FCAT) 
became a requirement for class of 2003, 
while passing the Regents Comprehensive 
Exams (RCEs) applied first to the class of 
2000, and the Competency Test in North 
Carolina went into effect in 1994. These 
assessments replaced minimum competency 
tests as high school graduation requirements 
in all three states. In addition, North 
Carolina high school students take a series 
of End-of-Course (EOC) exams that count 
for 25% of their course grade. California’s 
High School Exit Examination 
(CAHSEE) — the state’s first — was 
implemented in 2000-2001, and applied to 
the class of 2004 at the time of our 
fieldwork. In July 2003, just after the 
completion of our site visits, the State Board 
of Education voted to make the class of 



2006 the first students subject to the 
graduation requirement. 

These four states also held high schools 
accountable for student performance through 
a combination of rewards for high and/or 
improved performance and sanctions for low 
performance. Although the formulas for 
identifying schools differed across the states, 
all four took into account both the absolute 
level of student performance and changes in 
achievement over time. Low-performing 
schools received technical assistance (TA) 
through state assistance teams (NC, NY) and 
additional funds (CA, NY). Schools that 
failed to improve over a designated period 
of time could lose their students (FL), 
principal (NC), or accreditation (NY), 
and/or be subject to reconstitution (FL) or 
state takeover (CA). While the school was 
the primary target of accountability in these 
states, districts in North Carolina could have 
their superintendents and other 
administrators replaced and lose their 
accreditation if over half of their schools fell 
into the lowest performance category. 

Accountability policy in California, 
Florida, New York, and North Carolina was 
relatively stable. High school students in the 
last three states had been subject to a high 
school exit examination for many years. 
Florida and North Carolina’s school 
accountability systems were also at least a 
decade old, and could be characterized as 
“mature.” North Carolina’s ABCs program 
was enacted in 1994. Although Florida’s A- 
Plus program dates only from 1999, it 
refined and expanded an earlier 
accountability policy that focused on the 
state’s lowest performing schools. As 
California and New York’s school 
accountability policies are of more recent 
vintage, we have designated theme as 
“emerging.” California’s Public School 
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Accountability Act was passed in 1999. 

New York has held its very lowest 
performing schools accountable for several 
years under its Schools Under Registration 
Review program, but only extended its 
accountability policies to all schools in the 
state in 2000. 

Weak Systems 

While high school students in Michigan 
and Pennsylvania must also take state 
assessments, passage was not a state 
requirement for graduation. Students who 
passed the Michigan Educational 
Assessment Program (MEAP; Basic or 
above) received an endorsement on their 
diploma and could qualify for a college 
scholarship (score at Proficient or above). In 
Pennsylvania, local school districts 
determined whether to use the 1 l th -grade 
(Pennsylvania System of School Assessment 
(PSSA) and/or a local assessment as a 
graduation requirement. The state had 
considered, but not implemented, a policy of 
rewarding students who passed the PSSA 
with diploma seals. 

At the time of our fieldwork, Michigan 
did not hold high schools accountable for 
student performance. The state’s 
accountability system was initiated in 1994, 
but was placed on hold in 2000 while the 
state designed a legislatively mandated 
system based on multiple indicators of 
student performance and school context 
variables. State accountability ratings were 
not released until fall 2004. Michigan did 
not have a system of district accountability, 
either. Pennsylvania, however, held districts 
accountable for aggregate student 
performance. Under the state’s 
Empowerment Act of 2000, districts in 
which half of the tested students score 
Below Basic on the state assessment were 



subject to a series of sanctions, from the 
development of a district improvement plan 
to technical assistance to state takeover. 
Schools could receive rewards for improved 
test performance and/or attendance, but were 
not subject to sanctions. 

State Assessments 

High school assessments in the six 
study states varied on several dimensions: 
the number of assessments, grade level 
tested, content coverage, and remediation 
policy. All six states asserted that their tests 
are aligned with state standards. 

Three states — California, New York, 
and North Carolina — administer multiple 
tests to high school students that are, in turn, 
used for different accountability purposes. 
High school students, for example, take the 
CAHSEE in grade 10, the California 
standards tests (CSTs) in grades 9-11, and 
the California Achievement Test (CAT-6) in 
grades 9-11. While only the CAHSEE 
counts for high school graduation, all three 
tests are used to calculate a high school’s 
accountability index. Similarly, in North 
Carolina, only the eighth-grade End-of- 
Grade (EOG) exam is used for high school 
graduation. Scores on end-of-course (EOC) 
examinations count toward students’ course 
grades, and both of these exams, as well as 
the 10 th -grade High School Comprehensive 
Test, are used to calculate a school’s status 
and growth rate under the ABCs. The New 
York RCEs are given in multiple subjects. 
Students in the class of 2001 had to pass two 
exams (English and mathematics), and, 
starting with the classes of 2003 and 2004, 
students must pass five examinations (two 
additional in social studies and one in 
science). The other three states administer 
only one high school examination. 
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The high school exit exams are administered 
at different grade levels: grade 8 (NC), grade 
10 (CA, FL), and at the end of the specified 
course (NY). Similarly, EOCs in North 
Carolina are administered when students 
complete a tested course. Michigan and 
Pennsylvania administer ll th -grade high 
school tests. While the high school exit 
exams tend to be limited to 
English/language arts and mathematics (CA, 
FL, NC), high school students are also 
assessed in science and/or social studies in 
exams that count toward graduation and/or 
school accountability in California, New 
York and North Carolina. While not a high- 
stakes test at the time of our fieldwork, 
Michigan’s ll th -grade MEAP also covers 
science and social studies. 

The grade level and focus of the test 
(specific course versus more general skills 
and knowledge) also affect test content. For 
example, North Carolina’s exit exam covers 
the state’s eighth-grade standards, and 
students are expected to perform at grade 
level on that material to graduate from high 
school. California’s HSEE is aligned with 
9 th and 10 th grade English/language arts 
(ELA) standards and with sixth- through 
eighth-grade mathematics standards (which 
include algebra). Michigan’s ll th -grade tests 
cover algebra and some geometry, while 
Florida’s 10 th -grade and Pennsylvania’s 
ll th -grade mathematics tests include 
material through precalculus. North 
Carolina’s EOC exams assess the state’s 
Standard Course of Study in 1 1 courses in 
English (through English II), mathematics 
(through Algebra II), science (through 
Chemistry), and social studies. 

Finally, the four states with high school 
exit exams require that students who fail the 
tests receive remediation. Local districts 
design these remedial programs, which often 
receive additional funding from the state. 
Neither Michigan nor Pennsylvania fund nor 



require intervention services for students 
who fail their high school assessments. 

Standards: Specificity of 
State Guidance 

In 2002, the specificity of state 
guidance for the high school curriculum 
varied as well. At one end of the spectrum is 
North Carolina, with its course-specific 
guides and EOC examination structure. The 
latter provides teachers with specific 
feedback regarding student performance on 
a bounded content area, and gives school 
and district staff detailed information to help 
them adjust instruction to meet state 
standards. Florida’s curriculum frameworks, 
curricular planning tools, and course 
descriptions incorporate grade -by-grade 
state standards. California has high school 
standards that focus on grade clusters (9 th - 
10 th and 1 1 th — 12 th ), but more specific teacher 
guides to assist in preparation for the state 
high school exit exam. The latter generates 
subject-area scale scores and subscores on 
the skills and content of that test. By 
contrast to these states, the guidance offered 
by Michigan and Pennsylvania is much less 
specific. For example, both states’ standards 
documents and curriculum frameworks were 
only for benchmark grades, not for each 
high school course or grade level. 

Knowledge of State 
Accountability and Its 
Perceived Value 

The preceding discussion reveals just 
how complex and multifaceted some state 
accountability systems can be. Nevertheless, 
a majority of high school teachers across the 
states could paint at least a broad portrait of 
state accountability. Most had a clear 
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understanding of the potential consequences 
(or lack of consequences) in the system, and 
the elements for which they were directly 
responsible, such as student testing or 
documentation of curriculum alignment. 
Misconceptions or confusions about 
accountability were more common among 
educators in Michigan and Pennsylvania, 
where the accountability policies were in 
flux. For instance, Pennsylvania teachers 
were confused about the status of diploma 
seals, which the state had rescinded after a 
year, and did not understand the sanctions 
and rewards. Notably, Pennsylvania, unlike 
other states, applied accountability 
consequences primarily to districts rather 
than schools. 

Predictably, staff roles and 
responsibilities mattered in terms of the 
specificity and depth of their knowledge 
about accountability design. Principals, 
school improvement team members, and in 
some cases department chairs had a more 
complete and detailed picture of the system. 
District officials understood its complexities 
well. Of course, school and district 
administrators are usually responsible for 
submitting accountability data and reports to 
the state, and must answer to local school 
boards about schools’ progress, so it is not 
surprising that they have more intimate 
knowledge of accountability. 

As noted earlier, we were also 
interested in our educators’ opinions about 
their state accountability system, 
anticipating that these views would shape 
the extent to which they responded. In what 
follows, we discuss their views of each 
different component of accountability. 

Standards 

Educators’ views about their state 
content standards differed across the states. 
In California, North Carolina, and 



Pennsylvania, at least , 1 content standards 
appeared more highly regarded than the 
states’ testing or accountability programs. 
For example, the majority of North Carolina 
teachers believed that state standards 
accurately reflected what students should 
know and be able to do.” Many said that the 
state’s Standard Course of Study was the 
foundation for their teaching, and one 
referred to it as her “Bible.” Teachers even 
in nontested subjects used the state’s 
Standard Courses of Study as a guide for 
their instruction. 

In addition, teachers argued that content 
standards produced a more coherent 
curriculum and more consistent 
expectations. Said one California teacher: 

No, I really don ’t, I really don ’t think it’s 
[standards are] a negative. / can look at 
the standards and I can say, yeah, kids 
should know that. And we just can ’t go 
through education with a hit-and-miss 
thing, where one kid gets into a 
particular teacher’s class and learns a 
lot, [inaudible] another class and learns 
nothing except for what that teacher did 
over the weekend. And it’s got to be 
more, if we’re going to experience gains, 
it has to be more than just a hit-and-miss 
thing. (Teacher, San Antonio High 
School, CA) 

At least rhetorically, standards were 
often viewed as more legitimate goals for 
teaching than tests. One New York 
department chair, echoing a common 
sentiment in the school, stated emphatically 
that “lessons should be standards based. 
We’re not teaching to the test. 



1 Educators in the other states did not express 
much opinion one way or another about their 
state standards. 

2 There was one exception to this sentiment; 
Maple High School teachers did not think the 
standards were not adequate. 
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We’re teaching to the standards” (English 
Department Chair, Nelson High School). 

Interestingly, however, this school did 
conduct quite a lot of test prep activities. We 
found this schism between rhetoric about 
testing’s value and action elsewhere, across 
schools with relatively weak and strong 
performance under their accountability 
systems. 

State standards did not go without 
critique, of course. In Michigan, state 
standards for benchmarked grades were seen 
as too vague to provide useful guidance. 
(Indeed, the state, regional Intermediate 
School Districts, and school districts 
themselves were making an effort to 
articulate and specify the standards for 
classroom teachers.) Criticism of content 
standards was common in Pennsylvania, 
even though standards fared better than 
other components of the state accountability 
system. Pennsylvania has arguably had a 
more difficult political history of standards 
development than other states in this sample. 
For example, when the state first introduced 
standards in the late 1980s and early 1990s 
under the name Outcomes-Based Education 
(OBE), it galvanized opposition from all 
ends of the political spectrum. The Christian 
right community rallied against values 
statements. They were joined in opposition 
by more liberal groups who were concerned 
about the state asserting authority over 
curriculum, and bitter debates ensued. So, 
for example, staff in Orthodox High School 
and its district administrators thought the 
standards tainted by politics, and preferred 
national content standards. Support for 
standards across Pennsylvania schools is 
best characterized as moderate. 

Consequently, relatively few 
Pennsylvania respondents identified meeting 
standards in their goal statements, or in what 
they felt accountable for. Such results stand 



in sharp contrast to California, where 
teachers in six of the eight schools felt 
accountable for covering state standards. 
Indeed, in most other states, teachers 
typically said they felt accountable for 
meeting academic goals and/or aligning 
their curriculum to state standards, along 
with creating a love of learning and helping 
students to achieve their full potential. 

While improving student motivation, student 
behavior, and persistence in school were 
mentioned, they were not as prominent as 
the academic goals. 

It is interesting to recall that when 
contemporary standards were introduced 10 
or more years ago, they often met with 
extreme resistance even in states with a 
well-established history of curricular 
guidance. For example, there was a 
firestorm in response to New York’s social 
studies standards in the late 1980s (see 
Massell, 2000). In other states like 
Colorado, educators and the public were 
quite wary about the extension of state 
control into the curricular prerogatives of 
local districts and schools. But while the 
road to state leadership in curriculum has 
sometimes been tumultuous, standards have 
become an accepted and legitimate feature 
of state policy in all of our states, with 
Pennsylvania as a slight exception. 

Testing 

State assessments garnered a substantial 
amount of comment, both positive and 
negative. Some educators believed that 
testing set clear goals for students, and 
provided them with a useful way to calibrate 
their expectations of student performance 
and evaluate the success of their own 
teaching (e.g., Southern High School, NC) 
or of their departments as a whole (Upstate 
High School, NY). Said one New York 
teacher: 
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I think it [the Regents exam] really puts an 
end to social promotion. You know and this 
is no knock against any teacher but you 
know sometimes we might be inclined to 
pass students who have been working reed 
hard and recdly don ’t understand the 
material. And now it’s basically saying you 
can 7 move on unless you show proficiency 
on an exam. (Teacher, Nelson High School, 
NY) 

While she believed the Regents exam 
compelled teachers to hold back failing 
students (buttressing the argument that 
standards set high expectations), another 
teacher admitted he was more likely to pass a 
student failing his coursework if the student 
had an acceptable Regents score. In either 
case, teachers do seem to agree that standards 
build greater consistency from teacher to 
teacher and school to school, as standards 
reformers have argued (U.S. Department of 
Education, 1994). 

But while educators acknowledged 
positive aspects of state testing, more often 
than not they challenged the quality of the tests 
or their impact on the school’s curriculum. 
Teachers and administrators across the sample 
states questioned the accuracy of state tests as a 
measure of student achievement, either because 
the reliability and validity of the tests were 
flawed, or because they thought that once-a- 
year tests did not adequately capture what 
students knew. The largest area of concern was 
the impact of testing on the content of the high 
school curriculum. 

New Y ork teachers and administrators 
were extremely negative about the Regents 
examinations, particularly in mathematics. 

They argued that the validity of these exams 
had declined and that the assessments had 
become more of an obstacle to be overcome 
than a measure of true learning (e.g., River City 
High School). One argued that the math exam 
was so heavily dependent upon reading that 
even brilliant math students could perform 



poorly (Teacher, Nelson High School). Others 
complained that the Regents format was 
mysterious and continuously changing, with 
sections weighted differently from year to year. 
Indeed, just after data collection, the state 
pulled the easier of the two math exams (Math 
A) because only 37% of students passed. The 
state subsequently allowed students to graduate 
and receive local diplomas if their teachers 
attested that they had met state standards. 

Some math educators in North Carolina 
also challenged the validity of some state tests, 
but in this case they thought student scores on 
the state EOC examinations were higher than 
they should be. 

Those tests aren 7 real indicative of what 
kids can do. If I have a student that’s getting 
Fs and Ds in my class, and they can come 
out with a B on that exam, I have a problem 
with the validity of that test — I’m not sure 
how they score them, you know ? And I have 
had that happen lots — kids who have gotten 
Ds and Fs because they do absolutely 
nothing, and I can see them getting a D on 
the end of course exam, but then come up 
with a B. I’ve even had kids come up with an 
A on it. (Teacher, Lincoln High School, NC) 

Pennsylvania teachers thought that neither very 
low nor high end students were well-served by 
PSSA. Some Michigan educators observed that 
their scores had fluctuated widely from year to 
year. They thought these shifts were due to 
unreliable scoring and scaling practices rather 
than changes in their instruction or their student 
population. They and others noted that state 
tests were administered too infrequently, with 
results returned too late, to be of much use for 
improving classroom practice, countering the 
oft-stated policy expectation that test data 
would be a critical lever for instructional 
improvement. 

Many high school educators across the 
states thought that state assessments had a 
negative impact on the content of their 
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curriculum, and observed a variety of 
narrowing effects. Nontested subjects, such as 
foreign languages, reportedly received fewer 
resources under the new accountability regime. 
Teachers in tested subjects argued that they had 
had to reduce their curriculum to topics covered 
by the state test; some felt that the state tests 
forced them to cover too much, too quickly, at 
the expense of diminished student learning and 
dampened instructional creativity. Another kind 
of narrowing was experienced by students in 
some of our Florida and California schools; 
these students were reportedly required to take 
the same or similar courses over and over until 
they passed state tests. An assistant 
superintendent in one of our Pennsylvania 
districts captured the spirit of these concerns 
when he said that the state’s focus on testing 
led to a “teach-to-the-test-at-the exclusion-of- 
the-meat-of-leaming program” (Orthodox High 
School, PA). 

Accountability 

Of course, state tests were the primary 
performance indicators used in state 
accountability programs, so some of the 
positive and negative comments about the tests 
had close parallels with educators’ views of the 
accountability measures and consequences tied 
to the tests. Administrators and teachers 
expressed the sentiment that the idea of holding 
high schools accountable for test results was a 
legitimate expectation and could be useful in 
motivating them to reflect upon the 
effectiveness of their practices for student 
learning. For instance, one Florida principal 
said: 

But we have to start someplace, and not to 
start would be more detrimental. So I feel 
for some of the schools who are really 
having a difficult time with this test area, the 
grading and all that. But I also feel like we 
have to have something in place to make 



sure that we provide what’s best for the kids. 
(Principal, Harbor High School, FL) 

Similarly, a Michigan principal said his staff 
would become complacent without 
accountability or their regional accreditation 
process (Principal, Smith High School, MI). 
Said one California school administrator: 

Are they progressing? Are they progressing 
fast enough? Are you presenting a rigorous 
curriculum? I mean, that’s another word 
that we were using a lot last year, and I like 
it. And, sure, you could be a real nice 
teacher and very nurturing, but are you 
rigorous enough? Are you having them write 
too much in journals, in their personal... you 
know, I think journal writing is good, but 
are you have them write expository essays? 
Are you providing enough challenging 
reading material? Are you as a department, 
and then are you as an individual making 
sure that you ’re teaching as much as you 
can, as quickly as you can, and as deeply as 
you can? It’s always that thing of coverage 
or depth, you know. (Assistant Principal, 
Urban High School, CA) 

But despite this recognition of positive 
potential, educators more often expressed 
concerns about unintended, ancillary impacts 
on students, teachers, and schools. They were 
especially concerned about the effects of 
accountability on student and staff motivation 
and their mission, as well as the technical 
details or fairness of existing or proposed 
accountability designs. 

Maintaining high school students’ 
motivation to leam and remain in school was 
the number one challenge mentioned by staff in 
a majority of our high schools. Teachers in a 
few schools thought state tests’ rewards or 
sanctions could improve students’ motivation 
to achieve. In Michigan, the scholarship dollars 
attached to the 1 l th -grade MEAP were said to 
have helped some students take the exam more 
seriously, but most thought this was a 
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sweetener for those already capable of pursuing 
postsecondary options but lacking the financial 
resources. One Pennsylvania teacher observed 
that test pressures had raised the importance 
and value of the high school diploma for many 
of her students. Test pressure had served as a 
bit of a “wake-up call” and had given her 
“more backing as a teacher” because now it 
was not just her saying students need to master 
the material but also “the state” (Teacher, 
Lakewood High School, PA). 

But educators in states that attached strong 
sanctions for students to the tests, such as 
promotion and graduation, worried that they 
had harmed the morale of the lowest achieving 
students, and possibly spurred more to drop 
out. In one North Carolina school, for example, 
the assistant principal was deeply concerned 
about the “lie about dropouts” — the fact that 
the rate of 7-8% is reported to the press, but in 
actuality they regularly lose half of their 
freshmen class. He thought the EOC tests 
contributed to a downward spiral of failure. 

The assistant principal explained the scenario: 

These kids come in, and they have to pass 
this test to get through the course. They get 
a little behind and a little further behind, 
and they look, and they say, “I’m never 
going to pass this test, ” and they drop out. 
(Assistant Principal, Lincoln High School, 
NC) 

Similarly, a math teacher at Grant High School 
in North Carolina, who generally thought that 
the EOC strategy helped students to maintain 
their grades also said: 

[But] I’m not a hundred percent supportive 
of all the students being held accountable, 
and say they’ve got to pass Algebra 1 to 
graduate from high school. Some of them 
can ’t do it. Do we lose that kid to dropout 
just because he’ s frustrated with the 
requirement, or do we teach him a trade that 
he can be productive in society? 



Michigan educators thought that testing 
geometry was too much to ask of all students, 
particularly the non-college-bound. Similarly, 
teachers from River City High School in New 
York thought that high-stakes testing 
aggravated student apathy because it forced a 
diet of irrelevant curricula: 

The new mandates coming down from the 
state make me crazy. On the one hand, we ’re 
supposed to give kids authentic learning 
experiences, but more and more we ’re 
forced to teach to a test. They don ’t 
translate to anything real meaningful in 
their lives. (Teacher, River City High 
School, NY) 

Counter to policy intent, Florida educators 
felt that students did not care how they did on 
the exams required for graduation because they 
were permitted to retake the tests so often. 
Policy churn could also lower teacher and 
student motivation. 

California educators anticipated that the 
state would postpone or cancel the use of 
CASHEE as an exit exam, leading teachers in 
Urban High School to dismiss the exam. In 
addition, some California teachers perceived 
that students did not care much about the state 
Standardized Testing and Reporting (STAR) 
test either: 

But it’s definitely true that I’ve had classes 
where we ’ve passed out the STAR test and 
we’ve had kids bubbling C all the way down 
and put their head down for the rest of the 
class, and multiple times. (Teacher, Arnold 
High School, CA) 



2 

Indeed, after we were in the field, the State Board of 
Education postponed the effective date of using the 
CASHEE as an exit exam from 2004 to 2006, to ensure 
that students had an adequate opportunity to learn to the 
standards. 
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Teachers suspected that the large number of 
California state tests lowered students’ 
motivation to do a good job on them. 

The State Board of Education in North 
Carolina had many discussions about 
appropriately balancing pressures on schools 
and teachers versus students. Their decision to 
make the EOC tests one quarter of the final 
grade was an attempt to respect teacher 
judgment but also to give students an incentive 
to take the EOC seriously . 4 Nevertheless, 
educators in one struggling North Carolina 
school (Lexington High School) said that 
students did not care enough about the EOC 
tests 5 because they could still pass their courses 
without doing well on the test. But since the 
public relies on the EOC results to make 
judgments about the school, Lexington teachers 
felt considerable stress. The principal suggested 
that good teachers had quit as a result of the 
pressure, and staff morale was perceived to be 
low in other North Carolina schools. The 
morale problem is compounded in low- 
performing schools that lack a cohesive 
professional community. For instance, in Urban 
High School in California, where teachers did 
not regularly communicate about instruction 
and dialogue was infrequent, the accountability 
system seemed to reinforce low staff morale: 

Well, there ’s always, it always puts pressure 
on you. . . . The public ’s putting pressure on 
you, and saying you 're not good enough. . . . 
In a lot of cases, it has a deleterious effect 
on how people teach . . . it’s demoralizing. 
You 're doing everything you can, you 're 
working as hard as you can with the 
students you have sitting in front of you, 
recognizing where they are and trying to 
pull them along as fast as you can to where 



4 Correspondence from Charles Thompson, Professor of 
Education at East Carolina University (personal 
communication, October 18, 2004). 

5 Note that we did not interview teachers responsible for 
the eighth-grade basic skills test, which is required for 
graduation. 



you ’d like them to be. The implication that 
nobody had any standards before they came 
up with these things is insulting. (Teacher, 
Urban High School, CA) 

As with testing, many perceived that the 
design of state accountability had narrowed 
their traditional missions. While state 
accountability programs include student 
dropout rates and other measures, testing is 
given the largest weight in determining 
schools’ progress. Educators in our schools 
thought that the measurement of their success 
in these terms had damaged their traditional 
vocational missions, special emphases on the 
performing arts, or other goals. For instance, 
the principal in Roberts High School in 
Michigan implied that the devaluation of their 
vocational mission had harmed their strong suit 
as a school, as well as students’ employability 
after graduation. 

Finally, educators from many schools 
viewed their accountability measures, 
particularly the heavy reliance on subject 
matter tests, as an unfair and illegitimate way to 
judge the success of their school. Sample 
schools reported that many of their freshmen 
entered with very low reading abilities and with 
many other academic and social problems. Said 
one Florida math teacher: 

It’s really not fair to compare the school 
with other schools, if you know what I 
mean. If they switched — like, what they call 
A schools, if they switched out the faculty 
here with the Fs, with the D-school faculty, I 
know they wouldn ’t bring it up to an A and 
we wouldn ’t bring them down to a D. You 
just have different clientele, you 
understand? (Math Teacher, Oceanside 
High School, FL) 

They felt that students were so far behind in 
skills that the state tests were not a good 
measure of the progress they had made with 
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those students (e.g., Nelson High School, NY). 
For some schools, simply getting students to 
attend class was an important success, one not 
accounted for by their state’s emphasis on 
testing. Finally, administrators and teachers 
also expressed concerns over the fairness and 
accuracy of comparing their results to those of 
other schools, particularly when funding 
disparities between schools remained, or since 
schools served students with very different 
socioeconomic backgrounds. 

Salience of State 
Standards, Testing, and 
Accountability 

We anticipated that educators’ perceptions 
about the value of the state system would give 
impetus to — or, if negative, depress — the 
pressure they felt from and/or the extent to 
which they responded to their state 
accountability system. While this held true in 
some specific instances, we found major 
disconnects between educators’ views on the 
merits of the state system and its salience in 
high schools. 

For example, while state content standards 
had long been a fixture of the policy 
environment in California and were perceived 
as highly legitimate, one of the California 
schools in our sample was only just beginning 
to align its curriculum to standards. A number 
of California schools did not have any 
processes in place to secure alignment to 
standards, and implementation was reportedly 
mixed. We found a similar situation in some of 
our Florida schools. At Atlantic High School, 
the principal — an advocate of standards — 
thought veteran teachers were resisting the 
integration of standards into the school’s 
curriculum, and had to wait for retirement to 
build a staff more willing to buy into standards. 
As this suggests, leadership was crucial. For 
instance, district leadership greatly enhanced 



the prominence of standards in Lakewood and 
Mountain View High Schools in Pennsylvania. 

In North Carolina, the design of the 
accountability system gave standards 
substantial clout. There, course-based standards 
and course-based tests provided educators with 
strong guidance and motivation to use that 
guidance. This “seamless system” also fit 
nicely into the organizational structure of high 
schools, because it did not require as much 
work for high school teachers to determine who 
would teach what knowledge and skills as in 
states that had benchmarked standards without 
more specific guidance documents. 

In contrast to standards, and despite a high 
volume of criticism and concern, state tests 
were most likely to generate pressure and high 
levels of activity. Test-related initiatives ranged 
from more ancillary test preparation activities 
to the adoption of whole courses specifically 
designed to address skills and content on state 
tests. The response in New York best illustrates 
the point that negative views about testing did 
not necessarily depress school level action. 

New Y ork educators expressed angry 
challenges to the design and impact of their 
state testing and accountability programs; 
indeed the amount of criticism there was 
notably higher than in other states. 
Nevertheless, schools undertook a significant 
amount of action to address measured 
performance, above and beyond such state 
mandates as remediation. Educators in Nelson 
High School, for example, thought the Regents 
exams were not good measures of students’ 
content knowledge. But even though the 
school was listed as “satisfactory” on the state 
accountability index, test-related action was 
abundant: the English and mathematics 
departments consistently included Regents 
questions in classes, conducted test preparation 
and remedial activities, used state test data, and 
aligned their textbooks as well as summer 
reading lists to state standards and tests. 
Teachers also reported raising student course 
grades if students passed the Regents exams. 
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In Michigan, schools with the highest 
volume of criticism were the most responsive 
in their state context. For example, Roberts 
High School staff argued that MEAP and 
standards were not well aligned, did not map 
well onto students’ abilities and development, 
and had sharply devalued their vocational 
mission. Nevertheless, they systematically 
planned changes in instruction to respond to 
MEAP, including research paper assignments 
for MEAP, test-taking skills seminars, practice 
tests, MEAP skills in student planners, and 
curriculum alignment. Interestingly, compared 
to several other Michigan schools, the staff we 
interviewed met and talked frequently to one 
another about professional matters. Teachers 
also expressed a high level of trust for their 
school administrators. This apparently strong 
professional culture may explain the unusually 
high level of response here and in another 
Michigan school. 

Many administrators and teachers talked 
about using state and other test data to make 
changes in their curriculum, consider student 
placement, and the like, although use appeared 
to vary widely from school to school, and state 
to state. Some teachers and even principals in 
the weakly responsive Michigan schools 
admitted that they had never seen MEAP 
results, or had only read about them in the 
newspapers. By contrast, the majority of the 
teachers we interviewed in North Carolina 
looked at and used the results of their students’ 
EOC exams to target areas where they as 
individuals needed to improve. For example, 
one math teacher in Grant High School noted: 

One of the first strategies that I use is from 
year to year, 1 take that testing data, which 
...breaks down those objectives into specific 
objectives. And I can look at my class ’s 
achievement for those individual objectives, 
and I can pinpoint areas that I need to 
improve. I take my lowest objectives and 
those are the ones I make modifications in 
the structure. One year it was radicals. And 



so the next semester I did a lot more with 
explanation of radicals. 

The mathematics department in that same 
school used the results of the EOCs to realign 
their curricula, particularly in Algebra 1A and 
IB, and to require that students earn a grade of 
C before taking the next level of mathematics. 
The gulf in data use between a state like 
Michigan and North Carolina is explained by 
the fact that in Michigan, state test results are 
released once a year, but only for (primarily) 
ll th -grade students, and until recently the state 
provided no item skills analysis. Some 
Michigan educators thought the MEAP data 
were not trustworthy or useful for diagnostic 
purposes. In North Carolina, the data were 
course- specific and timely, and were often used 
by individual teachers. 

While high-stakes environments did 
generate more press in general, we found that 
even there the vast majority of high school 
teachers did not experience any direct 
consequences for success or failure, and most 
did not think that administrators really knew 
what was going on in their classrooms. 
Administrators or department chairs in many 
schools collected lesson plans and required 
teachers to document curriculum alignment to 
standards; in some cases, districts or states 
(e.g., Michigan) mandated these activities. 
However, more often than not, teachers did not 
receive feedback on these lesson plans, and 
teachers’ instruction was rarely monitored or 
evaluated outside formal tenure and evaluation 
requirements. Department chairs did not have 
the power or authority to observe teachers, 
remove teachers, or mandate instructional 
change, and those who did foray into changing 
their colleagues’ instructional practice did so 
delicately, trying to mask their efforts. As a 
result, even in high-stakes environments like 
Florida, teachers did not feel as though anyone 
was holding them accountable for the 
performance of their students or their school. 
Teachers’ work remained largely uncoupled 
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from the system of sanctions and rewards. Said 
one California teacher: 

No one ’s ever come in here and said, your 
test scores are too low, what are you doing 
about it? If you don 7 improve, you ’re going 
to lose your position here ... we ’re working 
in a really difficult neighborhood with a 
difficult student population, but when, hmm, 
but when it comes down to it, honestly, there 
... I don 7 feel threatened that I’m going to 
lose my job if enough of my students don 7 
pass the high school exit exam when they get 
to be a senior. (Teacher, Arnold High 
School) 

Nevertheless, teachers in both weak and 
strong accountability states articulated feelings 
of responsibility for their students’ test scores, 
as well as other academic outcomes — getting 
their students ready for the next course level in 
their subjects, and postsecondary futures (i.e., 
college or work). Teachers’ feelings of 
responsibility about test results tended to come 
from their own sense of professional obligation, 
concern for student success, responsibility to 
their colleagues or community, and/or the 
articulated concerns of their principals about 
school test results. For the most part, principals 
reported much higher levels of stress about 
testing results than their teachers, and 
communicated these concerns to their staff but 
did not actively manage incentives or day-to- 
day instruction to meet these goals. 

In addition to the pressures of student 
achievement outcomes, principals in some of 
our high schools questioned staff 
about course failures. Attendance and student 
dropouts were acknowledged concerns in these 
buildings; in one Michigan school (Jones High 
School), for example, the principal noted 
freshman failure rates of 50%. Some teachers 
said their principals took them to task on these 
numbers, and felt that they had to justify these 
grades, or alternatively to find ways to help 
these students pass. One teacher complained 



that students in her “so-called” required senior 
course were allowed to substitute other credits 
to graduate. This illustrates the dual and 
sometimes competing goals of accountability, 
which measures school success both on 
achievement and attainment measures. In 
educational practice, these goals often conflict 
and pose a tough challenge to high school 
teachers and administrators. 

We expected that we would find more 
consistent levels of response in high-stakes 
accountability environments, but this was not 
borne out. Instead, we found considerable 
variation in the salience of the system even in 
those strong accountability states, regardless of 
schools’ relative location on the accountability 
index. So, for instance, while our lowest 
performing schools were more responsive to 
external accountability than some research 
would have predicted given their circumstances 
(see Elmore, 2003; O’Day, 2004), we also 
encountered some very low-performing schools 
where addressing state accountability was a 
distant concern. For example, teachers in two 
of the three priority schools in our North 
Carolina sample reported feeling less press to 
address accountability than teachers in the third 
school. 

The evidence about the salience of the 
accountability system leads us to five main 
points: 

1. Examples of the disjuncture 
between acceptance of the 
components of accountability and 
levels of action in high schools 
suggests that the press of the system 
was getting through, despite 
significant doubts and concerns 
about the impact on teaching and 
learning. Response without 
conviction runs contrary to the 
conclusions of a body of 
implementation literature which 
states that action occurs after 
“street-level bureaucrats” become 
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committed to and engaged with the 
reform agenda (Lipsky, 1980; 
McLaughlin, 1990). 

2. While the consequences of 
accountability failures or successes 
on two actors in the schools — 
principals and students — were 
usually clear and direct, the 
consequences for teachers were not 
well articulated. (See also Goertz, 
2001 .) 

3. Nevertheless, teachers did focus on 
academic outcomes out of a sense of 
professional responsibilities that 
may have been accompanied by 
perceptions of more informal 
pressures from their administrators, 
colleagues, or the community. 

4. We found substantial differences in 
the extent to which state testing was 
salient to schools in the strong 
accountability systems of CA, FL, 
NC, and NY compared to schools in 
the weaker systems of MI and PA. 
While schools in the latter two 
states did address tested 
performance, their accountability 
systems were simply not as salient 
or pressing. This occurred even 
though the public reporting of 
MEAP scores had been part of the 
policy scene for many years — 
pointing out that often, public 
reporting is not enough to focus 
attention (for contrasting findings 
on the effects of public reporting, 
see Goldhaber & Hannaway, 2004). 
Certainly states with higher stakes 
had a legal obligation to students to 
provide more test-related 
opportunities and services. But 
again, these legal obligations did not 
explain all the types of responses 
occurring in these high schools. 



5. Despite this general pattern of 
higher salience in the high- stakes 
environments, it is extremely 
important to point out that the level 
of response within states was not 
consistent. In fact, we found as 
much variation in strong 
accountability states as weak ones. 

It is to this issue that we turn in the 
next section of the chapter. 

Explanations for 
Variation in Schools' 
Response to External 
Accountability: The Pull 
From Below 

Several factors seem to account for the 
wide variation in high schools’ response to 
external accountability. District-level 
leadership was critical, as were other local 
contextual factors such as community press, 
perceptions that ultimate consequences were a 
realistic possibility for the school, the school’s 
professional culture, teachers’ feelings of 
efficacy, and the school’s capacity to respond 
to accountability challenges. 

Active district leadership supporting 
accountability was associated with greater high 
school response inside all the strong 
accountability states, and even in Pennsylvania, 
with its weak and less stable system. 6 For 
instance, when the Renaissance City School 
District was placed on the state warning list for 
poor performance, the long-time superintendent 
decided that his earlier approach of delegating 



6 With rare exceptions (Hampton City and to a modest 
degree Foggy Mountain City), the districts in our 
Michigan sample were not very active vis-a-vis 
intervening in their high schools. During our fieldwork, 
public reporting of MEAP results was the only 
consequence in place for high schools or districts. 



31 




school improvement initiatives to the schools 
had failed. He started to recentralize control 
over the schools, and held them accountable for 
raising PSSA scores. He said, “The new 
number one job of administrators is improving 
student performance in reading and math.” 

The Lakewood High School principal in turn 
made the PSSA his primary focus and held his 
staff accountable for test scores. He met with 
teachers to discuss PSSA performance both at 
the whole school and department levels. In 
addition, he required teachers to use targeted 
PSSA workbooks. An English teacher said: 

1 can also feel the pressure that is on [the 
principal ]. Because when pressure is on him 
from the superintendent, which is an 
incredible pressure, I can hear it through 
what he says. You know, not directly, but 1 
can hear that pressure. And so it really 
keeps me alert and on my toes that, you 
know, you really need to be doing what you 
can to help him, which ultimately helps the 
scores. (Teacher, Lakewood High School, 
PA) 

Even low-capacity districts could stimulate 
higher levels of focus. For instance, one small 
North Carolina district with few central office 
staff used test score data to monitor school 
progress and signal the importance of student 
performance. The district also directed 
resources where needed, particularly to the 
lowest performing schools. 

District leadership could stimulate school 
action even when schools were relatively high- 
performing. For example, because its district 
was highly focused on test results and 
accountability, staff in relatively well- 
performing Southern High School (NC) 
expressed great fear about slipping into a lower 
rating. They discussed a variety of ways they 
used the EOC exams to drive their practice, 
including using test data to identify conceptual 
gaps and target students for extra help. 



Geography and policy design, along with 
district size, could influence district 
intervention on behalf of accountability. In 
Florida, for instance, districts focused on low- 
performing schools more than those in higher 
accountability categories, in part because state 
law required them to provide special assistance 
to these schools. But Florida district staff were 
stretched thin by the large number of schools in 
their jurisdiction. County administrators in two 
of our sample districts had 30 and 38 high 
schools, respectively. Even the smaller rural 
Florida counties managed five to six high 
schools — the same number of schools as our 
largest district in Michigan. The confluence of 
these factors led educators in our middle- 
performing Florida schools to respond less 
actively to state accountability. 

But district press was not in and of itself a 
sufficient factor in schools’ engagement with 
external accountability; we visited several 
schools that resisted their districts’ leadership 
in this area. One of our California districts 
targeted three schools performing poorly on 
state measures. In addition to a stronger press 
for improvement, the district provided more 
professional development and support to these 
high schools. But this pressure and support did 
not seem to erase the view in one of the target 
schools, Arnold High School, that no 
consequences were really likely to befall them 
or their students. They did not feel much 
pressure from their poor performance. Staff at 
Urban High School also expressed this 
sentiment: 

The adults say a lot of things that never 
really happen, like you ’re not going to pass, 
you ’re not going to graduate unless you take 
this test . . . but somehow these kids wiggle 
and worm and it all sort of falls into place . . 
. there ’s going to be whatever it is going to 
come cdong and sabotage that exit exam. 
(Teacher, Urban High School, CA) 
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For Arnold High School, the lack of 
responsiveness to their district’s press on 
external accountability was an issue of staffs 
knowledge and feelings of efficacy. The 
principal reported that while he did not know 
how to meet the goal that all students could 
succeed at high levels, his strategy was to 
emphasize this belief and ask his staff what 
they needed to make it happen. Yet he was 
turning to a staff of young and inexperienced 
teachers: Only 66% were certified, and many 
were teaching on emergency credentials. The 
school’s Academic Performance Index (API) 
statewide score had not been above a 2 (toward 
the bottom of the accountability spectrum) 
since 1999, and staff had come to believe that 
they would never be able to change these 
dismal results. 

On the other hand, perceptions of efficacy 
at the school level could also produce 
resistance to district leadership. For example, 
Medal County, North Carolina, focused heavily 
on data and the EOC exams. Although their 
initiatives influenced practice in Lincoln High 
School (teachers reported the exams’ strong 
impact on their instruction, goals, efforts to 
search for improvement strategies, and course 
assignments), it did not have much effect in 
neighboring Maple High School. In this school, 
only the math department reported using the 
EOC scores to place students in classes. Such 
differences appeared to be due to the fact that 
Maple performed well on the state system, and 
had a relatively new, forceful principal who 
sought to buffer the school from the external 
pressures exerted by the district. He felt the 
school was doing well and did need not make 
substantial change. Further, staff thought that 
district efforts to measure and spur student 
achievement — like the state’s — were not good 
indicators of student learning, and in fact had 
led to lower standards for both teaching and 
learning. 

Community expectations about academic 
performance — or more precisely schools’ 
perceptions of their community and its 



expectations — appeared to be another 
important ingredient influencing the 
responsiveness to external accountability of 
some schools, like Redwood High School in 
California. Although this school had a high 
position on the API — especially compared to 
schools serving similar populations — its staff 
pressed extremely hard to improve their results. 
The school was located in a wealthy 
community with high expectations for its 
schools. In addition, it had committed school 
leadership, ample fiscal resources, and a strong 
collegial atmosphere, all of which contributed 
to a strong sense of internal accountability (see 
Debray, Parson, & Avila, 2003). Similarly, in 
Striver and Mountain View high schools in 
Pennsylvania, staff concerns about community 
perceptions’ clearly drove their desire to 
improve performance on the PSSA. 

Conclusions and 
Research Implications 

High school teachers and administrators 
were aware — often keenly aware — of the 
challenges posed by external accountability, 
and spoke of the unintended consequences that 
such systems could create for their teaching and 
curriculum or for student motivation and 
persistence in school. Many were deeply 
mistrustful of state tests or other technical 
aspects of the accountability design, and had 
serious doubts about whether testing and 
accountability, in operation, were working to 
improve their educational practice. 
Nevertheless, even the most skeptical acted to 
address the demands of their testing and 
accountability programs — indeed, sometimes 
these schools were the most responsive. On the 
other hand, though standards were well 
accepted in most states, they did not always 
lead to much focused action. While in part this 
has to do with the fact that some schools and 
districts had already addressed standards 
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reforms in prior years, others had never done 
so. 

These findings challenge the longstanding 
notion that implementers’ belief is a necessary 
prerequisite to action. Part of the explanation 
may lie in the fact that while educators 
questioned the immediate effects of testing and 
accountability, they also recognized its 
potential value to ultimately improve various 
aspects of schooling. In future research, 
simplistic notions that “belief follows action” 
should be replaced with more discriminating 
models of the relationship between these two 
domains. 

Certain features of accountability designs 
raised the likelihood of action: Stronger 
consequences for students and schools did, in 
general, yield greater press and response, 
particularly when the system was stable. But 
there was a notable exception to this rule. Even 
though accountability consequences rarely had 
direct effects on teachers’ employment, their 
professional identity, care and concern for 
students, feelings of efficacy about their ability 
to address the challenges posed by 
accountability, and concerns for their 
administrators and community coupled them to 
the goals of external accountability. 

Furthermore, while we found confirming 
evidence that strong and stable accountability 
stimulated higher levels of press and action to 
improve on accountability measures, we also 
found that schools were not consistently 
responsive in any state system, be it weak or 
strong. Nor were schools consistently 
responsive if they were in a particular 
performance level in the system. We did learn 
that high schools tended to be more active 
when their district leaders were focused on 
accountability. The early RAND studies in the 
1970s also identified districts as crucial players 
in school-level implementation, but this lesson 
was oft-forgotten in the policy world of the 
1980s and 1990s. At that time, state and district 
education agencies were seen as anathema to 
improvement; popular reforms, such as site- 



based decision making, were designed to 
bypass districts and give schools greater 
autonomy from their bureaucracies. The 
wisdom of harnessing districts to the reform 
agenda has been lately rediscovered, but it is a 
lesson worth reiterating in the name of 
improving the efficacy of accountability. 
Understanding why some districts press for 
high schools to address accountability while 
others do not — and how they can do so more 
effectively in high schools — is an important 
question for researchers and policymakers to 
continue to pursue. 

Finally, educators were strongly concerned 
about the narrowing of the traditional mission 
of high schools, as well as various kinds of 
curricular and instructional narrowing that they 
perceived. To be sure, a central goal of the 
standards reform movement has been, in fact, 
to rein in the extremely diverse high school 
curriculum and to pare down nonacademic 
courses. The comprehensive high school 
curriculum began to emerge in the early part of 
the twentieth century when Progressive 
educators decided to go beyond the classics to 
prepare the greater majority of students for 
their certain futures: marriage and motherhood 
for girls and work in the new factories for boys. 
Curricular differentiation in high school 
expanded during the 1960s, when educators 
tried to make the curricula more socially 
relevant and engaging, leading to the much- 
criticized “shopping mall” curriculum (Powell, 
Farrar, & Cohen, 1985). 

Efforts to bring a more academic focus to 
the high school curriculum began in the early 
1980s, with the landmark A Nation at Risk 
report, and extended to standards-based 
reforms accompanied by performance 
accountability. It is clear, however, that these 
efforts push against some deeply held values 
about the role of high school. As Feslie Siskin 
once wrote: 

High schools . . . are being asked to take on 

a new task — something they were not 
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designed to do — to prepare students for a 
defined minimum academic standard, and to 
get cdl students to graduate by achieving 
that standard. We have certainly not 
organized high schools so that cdl students 
would take the same content, or meet the 
same standards to graduate. In fact, 
comprehensive high schools were 
historically designed to do precisely the 
opposite . . . their design imperative has 
been to serve democratic purposes and 
accommodate diverse student populations by 
creating a wide range of programs, and a 
differentiated curriculum. (Siskin, 2003, pp. 
176-177). 

Our educators’ concerns about this more 
restrained academic focus reflect a deep 
disagreement or at least consternation about 
whether it can engage and better educate all 
students. Their comments also indicate that 
narrowing occurs in a variety of ways — some 
in the spirit of reform ideals, some not. The 
Florida school that eliminated special projects 
to allow more time for academic courses seems 
to meet reform goals. But when students retake 
the same courses over and over until they pass 
state tests, the academic purpose seems to be 
lost. Researchers should make a closer study of 
just what kind of narrowing is occurring, and 
how it may be influencing students’ persistence 
in school. 
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Table 1 . Key Characteristics of State Accountability Policies for High Schools, 2002-2003: Six Study 
States 





Target of 
accountability 


Assessment 


Accountability 

Measure 


Consequences 


Stability 


Strength of 

overall 

systenY 




Student 


CAHSEE: grade 10 


Students must score 60% in ELA and 
55% in math. 


Graduation: class of 2006; 
scholarship money. 


Unstable 




CA 


School 


CA Standards Test 
(CST), CAHSEE, 
CAT/6 


Academic Performance Index (API): 
based on performance and growth. 


Monetary rewards for growth. 
Sanctions for low performers: TA, 
outside intervention, and possible 
takeover. 


S table/Emerging 


Strong (4) 


FL 


Student 


FCAT: grade 10 


Passing score is middle of Basic (Level 2 
of 5). 


Graduation: class of 2003; 
Certificate of Achievement for 
higher score. 


Stable/Mature 


Strong (5) 




School 


FCAT: grade 10 


A+ Plan: based on performance, growth 
overall, and gains of lowest performing 
students. 


Monetary rewards for high 
performance and/or growth. 
Sanctions for low performers: TA, 
student choice, and reconstitution. 






MI 


Student 


MEAP: grade 1 1 


Performance level is Basic. 


Diploma endorsement. 


Stable 


Weak (1) 






Performance level is Proficient. 


Scholarship money. 






School 


MEAP: grade 1 1 


None. 


None (rewards?). 


Unstable 




NY 


Student 


Regents 
Comprehensive 
Exams (RCEs) 


Passing score of 65 (Proficiency) or 
higher for students entering ninth grade 
in 2001-2002; local option to set score at 
55 (Basic Proficiency) for prior classes. 


Graduation: class of 2000. 








School 


RCEs 


Performance Index (sum of percentage of 
students scoring above Basic Proficiency 
and percentage above Proficient). 


Rewards for high performance 
and/or growth (?). 

Sanctions for low-performers: TA, 
additional funds, and loss of 
accreditation. 


S table/Emerging 


Strong (5) 


















District 


RCEs 




May be designated Below 
Standards; develop improvement 
plan. 
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Target of 
accountability 


Assessment 


Accountability 

Measure 


Consequences 


Stability 


Strength of 

overall 

system 3 


NC 


Student 


8 th Grade End-of- 
Grade (EOG) Exam 


Passing score: achieving on grade level. 


Graduation. 


Stable/Mature 


Strong (5) 


End-of-Course (EOC) 
Exams 


Passing score. 


25% of course grade. 


School 


EOC; 10 th -grade 
Comprehensive Test 


Percentage of students passing EOCs; 
expected growth on EOCs, between 8 th - 
and 10 th -grade competency tests. 


Monetary rewards for higher 
performance and/or growth. 
Sanction: state assistance team, 
removal of principal. 


District 




Half of schools are in low-performing 
category. 


SDE can replace superintendent or 
other administrators; LEA can lose 
accreditation. 


PA 


Student 


PSSA: grade 1 1 or 
Local Assessment 


Proficiency on PSSA or local assessment 
as determined by LEA. 


Graduation. 


Unstable 


Weak (1) 


School 


PSSA: grade 1 1 


Increase PSSA scores by at least 50 
points, increase attendance. 


Monetary rewards. 


S table/Emerging 


District 


PSSA 


50% or more of students at Below Basic 
performance. 


Monetary rewards for improving 
performance of ELL, disabled, poor 
students. 

Sanctions for low performance: DIP, 
TA, additional funds, state takeover. 



A Rating of “Weak” to “Strong” determined by authors on the basis of target of accountability and strength of consequences. Numeric rating ( ) assigned by Carnoy and Loeb (2004). States assigned a rating of 1 
have state assessments but no school or student sanctions. States assigned a rating of 5 test students in multiple grades, strongly sanction and reward schools, and require students to pass a high school graduation 
test. States with strong school sanctions but no high school exit exam are assigned a rating of 4. 
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Table 2. Characteristics of State Assessment Systems for High 
Schools, 2002-2003: Six Study States 



CA 


CAHSEE (Grade 10) in EL A and mathematics. EL A aligned to 9 th /10 th -grade 
standards; mathematics aligned to sixth- to eighth-grade standards, including 
Algebra 1. Multiple choice format (two writing items). Students are tested in March 
or May. Students who fail have up to three retakes. New test implemented in 2000- 
2001. 

State funds summer school for students in grades 7-12 in danger of failing the 
exam. LEAs must provide remediation and supplemental instruction to students 
who fail. 




California Standards Tests ( CSTs) in English, mathematics, and science in grades 
9-11 and in history/social science in grades 10-11. Aligned to state standards. 




CAT/6 in reading/language skills, math and science in grades 9-11. Norm- 
referenced test (NRT). Becoming smaller component of state assessment system. 


FL 


FCAT (Grade 10) in ELA and mathematics. Standards-based and norm-referenced 
items. Standards-based mathematics items cover algebra and geometry and are 
aligned to Sunshine State Standards. NRT mathematics topics include algebra, 
geometry, trigonometry, and precalculus. Mixed item format. Test is given in 
March. Students may retake exam five times in grades 1 1 and 12. Replaced High 
School Competency Test in 1998. 

LEAs must provide remediation for students who fail exam. State provides 
Supplemental Academic Instruction (SAI) funds. 


MI 


MEAP (Grade 11) in ELA, mathematics, social studies, and science. Criterion- 
referenced tests. ELA has writing, reading and listening component. Mathematics 
covers algebra and some geometry. Test is given in January. Students may take in 
10 th grade and retake in 12 th grade. Revised in 2001-2002 (math) and ELA (2002- 
2003) to align with 1996 Curriculum Frameworks. Replaced High School 
Competency Test. 

Remediation is district/school option. 


NY 


Regents Comprehensive Exams (RCE) in English, Mathematics A for class of 2001. 
Subsequent classes must take two additional tests in social studies and one in either 
science or a foreign language. Students take at completion of course. Are aligned to 
state Learning Standards and to grade 9-12 courses of study. Students may take 
component retests for sections they failed. 

Schools must provide Academic Intervention Services (AIS) to students at risk of 
not passing exams; LEAs design programs. 


NC 


8 th Grade End-of-Grade (EOG) Exams in reading comprehension and mathematics. 
Aligned to state’s eighth-grade Standard Course of Study. Replaced Minimum 
Competency Test as graduation requirement in 1994. Students may retake exam 
through 12 th grade. 

Schools must provide remediation to students who fail eighth-grade EOG exams. 
Remediation for EOCs is local option and participation is voluntary on part of 
student. 
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Encl-of-Course (EOC) Exams in 11 high school courses ranging across 9 th to 12 th 
grades, including Algebra I; Algebra II; Biology; Chemistry; Economic, Legal, and 
Political Systems; English I; English II; Geometry; U.S. History; Physical Science; 
and Physics. Aligned to state’s course specific Standard Course of Study. Students 
take at completion of course. In place since mid-1980s. Became part of state 
accountability system in 1997-1998. 


10 th Grade High School Comprehensive Test in reading comprehension and 
mathematics designed to measure growth from 8 th to 10 th grade. Multiple choice 
format. No student consequences. 


PA 


PSSA ( Grade 11) in mathematics, reading, and writing. Writing also tested in ninth 
grade. Aligned to state standards. Mathematics coverage can go through calculus. 
Mixed item format. Students are tested in late March and can retake PSSA in 12 th 
grade. 

Remediation is local option. 
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Chapter 3 

Got You Under My Spell? 

How Accountability Policy Is Changing and Not 
Changing Decision Making in High Schools 

Betheny Gross, Michael Kirst, Dana Holland, and Tom Luschei 



Introduction 

The success of the current 
accountability movement, unlike many 
policies in the past that have mandated the 
use of specific materials, distribution of 
resources, or specific programs for 
improvement, relies almost entirely on the 
policy’s ability to prompt a response from 
schools and school’s ability to generate an 
effective response. Given the limitations on 
capacity building at the state level, states are 
relying heavily on the knowledge, skills, 
resources, and initiative of local agents for 
the desired improvement. State 
accountability policy will necessarily fail if 
local agents such as teachers and 
administrators are not compelled to 
acknowledge the state’s performance 
targets. The policy will also fail if local 
agents do not select and effectively 
implement strategies to improve student 
performance in both the short and long term. 
Given this reality for the state accountability 
policy, the process of decision making in 
schools characterized by the goals and 
problems around which individuals in 
schools focus their improvement efforts, the 
information they use in addressing these 
problems, and the nature of the strategies 
that they ultimately select are of critical 
importance to the extent of and nature of 
improvement schools will see in student 
performance. 



In this chapter we feature two goals (a) 
to reveal the influence states are having 
in the decision-making process of high 
schools through their accountability policies 
and (b) to give a general sense of the range 
of strategies selected by high schools under 
accountability. We pursue these goals with 
interview data collected from teachers in 48 
schools across six states; California, Florida, 
Michigan, New York, North Carolina, and 
Pennsylvania. These six states reflect 
variation in their accountability policies 
across the country before the 
implementation of the No Child Left Behind 
Act (NCLB). As raised in the introduction, 
this study focused on high schools because 
little work to date has been devoted to 
reform in high schools and because the era 
of accountability and standards calls for a 
significant departure for this institution, 
which has historically been noted for 
resisting change. The attention to decision 
making in this chapter, however, offers yet 
one more reason to examine high schools. 
Unlike elementary and middle school 
leaders, for whose institutions countless 
reform models have been designed and 
subsequently employed in efforts to meet 
accountability demands, high school leaders 
have relatively few models or school designs 
to which they can turn for guidance. High 
school leaders are very much left to their 
own experiences, knowledge, and resources 
to change organizations that have struggled 
with change. Given this limited guidance, 
this question about how accountability 
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policies focus attention and motivate change 
is all the more interesting in the high school 
context. Can accountability policy move 
people when resistance has been the history 
and guidance is limited? 

We begin the discussion of 
accountability and decision making in high 
schools with a theoretical discussion of the 
decision-making process, which we define 
as consisting of the five components 
introduced by Cohen, March, and Olsen 
(1972). These components include (a) the 
decision situation, (b) participants in 
decision making, (c) the decision maker’s 
goals and the problems addressed, (d) the 
information used in decision making, and (e) 
the solutions selected by schools. We follow 
this theoretical overview with an analysis of 
the data from our sample of 48 schools. This 
analysis is broken into two parts: (a) an 
examination of the context of decision 
making in high schools and (b) a look at the 
extent to which the decision process of 
schools seemed to introduce strategies that 
moved schools away from their traditional 
mode of operation toward approaches that 
could potentially change the educational 
experience of students in the school. For the 
latter discussion we turn to models of 
information, search, and selection found in 
organizational literature to learn what 
strategies might be predicted relative to what 
high schools seem to be doing. 

In our discussion of the decision 
process and the nature of solutions we show 
how the states we visited seemed to be 
playing a relatively important role in the 
schools we visited, particularly in creating a 
focus on goals. With only a few exceptions, 
the states have managed to influence the 
goals and sense of accountability felt by 
teachers and administrators. Schools also 
seem to be responding to their state’s 
pressure with a battery of improvement 
strategies that focus on reaching the 
performance targets. Despite this clear push 



toward the state’s goals and active response 
by schools, we found that schools by and 
large did not engage in reforms that 
represented the coordinated efforts of their 
staff, take advantage of common 
organizational structures such as 
departments and planning teams, or 
encourage extensive use of external 
assistance (even when external assistance 
was required by the state). We also found 
that the search for solutions in schools 
remained largely local, with the vast 
majority of information applied to decision- 
making efforts coming from within the 
school and most strategies offering very 
little challenge to the fundamental 
curriculum or practice of the schools. While 
our data revealed that local information and 
strategies dominated the decision-making 
scene in schools, our data also showed a 
handful of relatively effective avenues 
through which new ideas entered the schools 
and through which external change agents — 
particularly districts — may attempt to access 
schools. In addition, we found some 
situations in which these avenues of new 
information, particularly through districts, 
generated increasingly deep challenges to 
the traditional practices and curriculum in 
some schools, suggesting that high schools 
may not be as intransigent as commonly 
thought. 

Theoretical 
Framework of 
Decision Making 

The elements of decision making listed 
above — decision situations, participants, 
problems, and information — break down 
into two primary phases of decision making. 
The first phase in decision making sets the 
stage for the decision making and includes 
descriptions of the decision situations, 
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participants, and problems. These elements 
reveal how the process will engage, who 
will be involved, and what they will address. 
The second stage, which involves the search 
and selection of solutions, features the 
information used in decision making and the 
solutions that result. Here the decision 
maker seeks and scans information about 
possible solutions and selects a strategy 
expected to address the problem or problems 
identified. Examinations of decision making 
in organizations have illuminated a number 
of issues that decision makers both confront 
and create in these phases of decision 
making. This section offers a brief 
description and theoretical considerations of 
the decision-making components, which we 
have organized into the phases of setting the 
scene and search and selection. Instead of 
offering a complete review of this extensive 
literature, the sections that follow discuss a 
framework that has a specific focus on 
schools. 

Elements That Set the Scene 
for Decision Making 

Decision Situations 7 

Decision situations are the time and 
place for decision makers to engage in the 
process of finding a solution. Organizations 
reveal a relatively constant flow of 
opportunities for decisions to be made by 
the organizations, groups within 



7 Cohen, March and Olsen (1972) use the term 
“choice opportunity” to identify the forum in which 
decision makers are expected to produce some 
solution for a specified problem. However, the term 
“choice opportunity” suggests that the decision 
makers will most likely select a solution among 
alternatives, a process that is not always apparent. 

We have chosen to use the term “decision 
opportunity” to emphasize the decision being made 
over the process of weighing alternatives and making 
a choice. 



organizations, or individuals. Some of the 
decision opportunities recur at regular 
intervals, such as annual school plans, while 
other opportunities occur as a response to 
current circumstances. Decision 
opportunities need not involve a committee 
at all but, instead, may be informal as would 
be the case when an individual confronts an 
issue and finds a solution independently. 

Participants 

Participants or participant groups 
include standing committees and ad hoc 
committees as well as informal groups or 
individuals with authority over some aspect 
of the organization. A number of conditions 
determine who will participate in decisions. 
Certainly the amount of nonteaching time 
available to teachers limits their 
participation in decision situations. The 
physical proximity of teachers to each other 
during their occasional free time along with 
the personal relationships teachers share 
determines which teachers cluster for 
impromptu and informal decision situations. 
Policy mandates that require participation of 
teachers, community members, or 
administration purposely select participants 
for decision situations. There are likely 
dozens of conditions unique to each school 
that shape the participant list in any decision 
situation. However, organizational structures 
related to the organization of teachers, the 
distribution of power implied by hierarchy, 
and norms of teacher autonomy are worth 
noting specifically because of their 
commonality across schools. 

Subject departmentalization is one of 
the most common organizational features of 
American high schools and gives rise to the 
image of high schools as collections of 
intellectual “silos.” Teacher specialization 
and structural organization by departments 
proves useful for creating subcommunities 
with common focus, which has been found 
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in organizational science to be an effective 
organizational strategy for organizations that 
are comprised of groups with specialized 
knowledge (Bolman, 1997). However, this 
partitioning of teachers has led to 
organizations in which teachers have few 
opportunities to participate in decisions 
related to other departments in their school 
or coordinate schoolwide decisions. While 
the schools in our sample occasionally 
supplemented the traditional structure with 
cross-department teams or within- 
department subteams, only one school 
organized teachers into interdepartmental 
teams, and only one school that disbanded 
departments to discourage the balkanization 
of teachers deviated from the traditional 
department structure. 

The hierarchy of educational 
institutions offers a second structure that 
determines the participants in decision 
situations. The educational bureaucracy that 
exists today dates back to the 
professionalization of education in the early 
1900s (Tyack, 1974). This movement 
established a hierarchy of schools nested 
within districts and districts nested within 
state departments of education as well as the 
hierarchy of administrators governing over 
teachers. Due to the ties districts have to 
schools, district officials can and do 
participate in decisions by sending 
representatives to decision situations or 
simply operating through the influence their 
authority grants them. While it is rare for 
state department officials to sit on local 
committees through a decision process, the 
power states exert through the provision of 
funding and directives makes the state’s 
policies and preferences strong players in 
decisions at the local level. 

At the school level the power structure 
of administrators over teachers, coupled 
with the well-documented norm of teachers’ 
professional autonomy, creates a situation in 
which administrative staff assume 



considerable authority over schoolwide 
decisions while teachers enjoy relative 
freedom to make choices concerning their 
classroom and instructional practice. While 
efforts to standardize curriculum and guide 
curriculum goals through testing have 
limited some of the authority teachers held 
over their course content, and teachers in 
many cases reported their loss of influence, 
the teachers in our sample still reported 
control over some aspects of their 
instructional practice and approaches. 

Problems 

Problems are the issues and challenges 
that become the focus of the decision 
situations and represent the point in the 
decision process at which states aim with 
accountability policies. Problems are 
constructed by members of the organization 
and are a function of the organization’s 
goals, the goals of those in the organization, 
external expectations, and the local 
conditions that are perceived as impeding 
progress toward these goals (Cohen, March, 
& Olsen, 1972). States’ main objective with 
accountability policies is to directly 
influence schools’ goals and the 
prioritization of their goals. Each of the 
states had established both assessment 
targets and content standards that the states 
accompanied with a battery of rewards and 
sanctions. States expected schools to 
incorporate these targets and sanctions into 
their articulated goals and thus shape the 
problems they attempted to address. 

It is important to note the complexity of 
goals in organizations created because the 
many and varied school-level decision 
makers operate with somewhat different or 
even conflicting goals. The variability and 
incongruity of goals and identified problems 
often lead to fragmented and inconsistent 
strategies in organizations (Locke & 

Latham, 1990). While we recognize the 
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complexity of goals and problem 
identification as important to an analysis of 
solutions, we, unfortunately, could not 
explore the consequences of this complexity 
with much depth in this broad look at 
decision making. 

Searching and Selecting 
Solutions: The Use of 
Information 8 in Decision 
Making 

The heart of decision making occurs 
when decision makers bring information to 
bear on problems and select solutions. 
Information, the search for information, and 
the range of information used in the decision 
process reveal the potential for change. 
Because we are particularly interested in 
change and the potential for change, we 
focus on the source of information, the 
potential for new ideas to enter the school 
and introduce variation (Axelrod & Cohen, 
2000), and the extent to which the solutions 
impact the dominant structure and practice 
in the school, referred to in this chapter as 
the core technology of the school. 
Fortunately, literature on decision making 
offers useful guidance in considering the 
nature of information sought by and brought 
to decision makers as well as the 
relationships between the solutions that 
result from the search of information and 
current practice. 

Authors of decision-making theory 
explain that information exists within 



8 Cohen, March, and Olsen ( 1972) use the term 
“solutions” to refer to the ideas that potentially 
address the issues considered in a decision situation. 
We have changed the term “information” to create a 
distinction between the ideas that are possible. In 
addition, because I refer to information rather than 
solutions, this discussion can include information that 
is used to evaluate and assess a problem and possible 
solutions. 



organizations and enters the organization 
through a variety of sources (Brown, 1993; 
Huber, 1996; March, 1994). The members 
of an organization collect a great deal of 
information through professional 
development opportunities, their 
experiences, and their history of 
conversations with those in and outside the 
organization. As a result, organizations 
typically hold a vast array of information 
that rests latent in the organization and, 
therefore, hold the potential to assess or 
resolve problems. Information that is not 
held within the organization can be actively 
sought by looking outside the organization, 
soliciting information from external agents, 
passively receiving information from active 
information providers, or discovering 
information through research and 
development (March, 1994). 

Models of decision making take what is 
known about the sources of information and 
offer a picture of the process through which 
decision makers put information to use and 
select strategies. The model of decision 
making we find particularly relevant given 
the context of schools and the context of 
accountability is a model described as a 
bounded rationality model (March, 1994; 
Simon, 1986). This model, unlike traditional 
models of decision making, assumes that 
decision makers face significant limitations 
in (a) their opportunity to access or acquire 
information, particularly the wealth of 
information outside the organization, and (b) 
the extent to which reliable strategies exist 
and can be identified by the decision maker. 

Authors in the tradition of bounded 
rationality argue that information is not as 
easy to obtain as often thought. Information 
can be very expensive to obtain or difficult 
to locate. Information also takes time to 
locate, recognize, and process. Seeking 
information may simply take more time than 
decision makers have or allocate to the 
decision process. The conditions of resource 



47 




limitations constraining access reflect 
common situations in schools where access 
to information often requires that staff and 
substitutes be paid to give teachers and 
administrators leave to attend conferences or 
professional training. Many rural educators 
describe the lack of regional resources. In 
addition, the rapid nature of decision making 
in schools, a point that will be discussed in 
more detail later, leaves little time to explore 
the landscape of potential solutions. 

Constraints, however, are not only 
imposed by resource limitations. Decision 
makers can intentionally or unintentionally 
put up their own barriers to information. 
Teachers can actively resist or mistrust a 
certain set of ideas, particularly external 
ideas, and therefore not seek ideas outside 
the organization, block efforts to bring ideas 
into the school, or simply deny their use in 
their classrooms. Larry Cuban (Cuban, 

1993), in a study of teachers’ practice from 
1880 to 1990, argues that teachers, 
particularly high school teachers, appear 
very selective in their adoption of new 
strategies, selecting only a small set of the 
reforms proposed over the years. This 
selectivity has led to a relatively constant 
form of instruction in schools over this 
period, with norms of practice set firmly 
around traditional practice. Although Cuban 
does not argue that teachers staunchly 
defend their current practice and 
organization of their work, this scenario 
seems plausible given the resiliency of 
traditional practice. In more resistant 
contexts there is little incentive for school 
leaders to seek ideas outside the school or 
consider new ideas that approach or even 
enter the school. In addition to this intended 
resistance, teachers or administrators may 
not be “tuned into” ideas outside the school 
and, therefore, never realize that such an 
effort can be made. Each of these examples 
shows that (because of a variety of barriers 
internal and external to the school) access to 



information can be, and often is, very 
constrained. 

In addition to constraints on the access 
of available information, there is also the 
question of whether information on potential 
strategies even exists. The process of 
teaching and learning is commonly thought 
to be what organizational theorists refer to 
as “complex technology” with unclear and 
unreliable strategies as hallmarks of 
complex technologies. For example, several 
teachers we spoke with challenged the 
notion that we know how to effectively 
teach adolescents how to read. Educators, 
perhaps partly in response to inconsistent 
evidence from research, are often skeptical 
that good strategies exist or that strategies 
used elsewhere can work in their unique 
context. In research by Corcoran (Corcoran, 
2003) on the use of research-based practices 
in schools, he found that district-level 
decision makers, who ostensibly have more 
opportunity and time to seek strategies than 
the typical high school teacher or principal, 
expressed frustration with the level and 
quality of research done on the issues that 
pressed their schools the most. Furthermore, 
the research that was available to these 
district officials often confused the issue 
with conflicting results. Perceptions that 
little information exists to be found may 
very well reduce the likelihood that 
information will be sought and that new 
strategies will be found. 

Given the constraints on access and the 
perception that only a few or no reliable 
strategies can be sought, proponents of 
bounded rationality argue that it simply 
makes no sense for decision makers to 
engage in a wide search of all, or most, 
possible strategies then select the best 
among them. 9 These authors instead suggest 



9 Scanning all possible strategies and then selecting 
the best among the possibilities is known as 
optimizing, the decision rule found in rational 
theories of decision making (March, 1994). 
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that decision makers under these contexts 
identify a target and seek only to meet the 
target with searches that are relatively 
limited in scope (March, 1994; Simon, 

1986), a condition often referred to as 
geographically local search (Rosenkopf & 
Almeida, 2003). The limited searches serve 
to save resources and reflect the constraints 
on access due the many reasons stated 
above. In addition, this limited search 
reflects the reality that unreliable technology 
leads decision makers to privilege strategies 
already used in the organization, use the 
existing structure of the organization, and 
leave the work of the organization intact, a 
well-documented condition referred to as 
technologically local search in studies of the 
private sector (Rosenkopf & Almeida, 



2003). Consequently, decision makers who 
focus on targets and limit their range of 
search tend to rely heavily on local 
information and select strategies that 
maintain the core technology of the 
organization. This model of search and 
selection is reflected in Figure 1. 

While this model of information, 
search, and selection is common across 
organizations and seems relevant for the 
school setting, some researchers have shown 
that organizations will reach beyond their 
borders and pull in strategies that change the 
fundamental practices and structure of the 
organization. In the rest of this chapter, we 
explore the extent to which schools’ 
responses seem to parallel this model or 
deviate from this model. 



Fi gure 1 . Model of Information Search, and Solutions 
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Setting the Scene for 
Decision Making: The 
Context of Decisions in 
Schools 

We begin our discussion of decision 
making in high schools with the elements of 
decision making that set the stage for 
decisions. Even though all three of the 
components — decision situations, 
participants, and problems — are important to 
understanding the context of decision 
making, we found accountability’s influence 
most clearly in the respondent’s 
identification of problems. We, therefore, 
begin this section with an extensive 
discussion of the goals and problems 
identified by individuals in the schools. In 
this discussion we show that in many ways 
people in schools are responding to their 
state accountability systems. However, as 
our discussion of decision situations and 
participants explains, we found that the 
dependence on ad hoc, individual, and 
uncoordinated decision situations suggests 
that schools in large part have not generated 
a truly organizational response to their state 
accountability systems. This relatively 
informal and independent nature of most 
decisions being made in high schools 
continues, despite the growing interest in 
schoolwide reforms that attempt to 
coordinate decision making and state efforts 
to require lower performing schools to work 
with teams or partners to coordinate school 
planning. 

Problems 

While a great many experiences and 
context conditions influence the construction 
of problems in organizations, goals 
theoretically play a central role in 



determining when and how problems are 
identified (Scott, 1998). These goals help 
organizations target their attention by 
clarifying what should be accomplished and 
highlighting the issues that impede progress 
toward the goals (March, 1994; Simon, 

1986) and signaling when the organization 
needs to make changes. Thus, goals help 
determine when problems need to be 
addressed and help to clarify what those 
problems must be. It is precisely at this point 
in the decision-making process that states 
exert their most direct influence with 
accountability policies. Although states 
seemed to modestly influence the mechanics 
of decision making by requiring decision 
situations and participants in low- 
performing schools and indirectly 
influencing district involvement in schools, 
goal setting, backed by incentives, is the 
central means through which states attempt 
to influence the decision-making process in 
schools. 

During our visits, we engaged 
respondents in two sets of questions related 
to goals. First, we asked respondents to give 
us their goals, the goals for their 
departments. Second, we asked for what and 
to whom they feel accountable. Together 
these questions revealed the goals they most 
immediately identified with and those goals 
they felt responsible for meeting. The 
teachers’ and administrators’ aims revealed 
that school-level respondents consider a 
variety of goals. Among the most common 
goals across the schools we visited were 
goals for student attainment (high school 
diplomas and postsecondary education), 
advancement in learning, and social and 
intellectual development of students. Of 
particular importance to this study, however, 
is the extent to which the state’s goals — set 
by performance targets and sanctions — 
influenced teachers’ construction of goals 
and accountability. 
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To a large extent states seemed to be 
effective at capturing schools’ attention and 
focusing teachers and administrators on their 
performance goals. Respondents in most 
schools in our sample incorporated state 
goals into their goals and sense of 
accountability, but these goals did not push 
out all of the traditional goals typically 
offered by teachers or based on teachers’ 
own work and experiences. Across all six 
states, at least some teachers in 35 of the 48 
schools in our sample described goals or 
accountability responsibilities in terms of 
their state’s standards and/or assessment 
(see Table 1). Not surprisingly, respondents 
in 15 of the 16 schools in our sample from 
the two of the highest accountability states, 
North Carolina and Florida, mentioned their 
state standards or assessment when 
articulating their goals and accountability. 
Interestingly, however, stakes are not 
everything in determining the influence of 
the state’s policy. Despite very low stakes in 
Pennsylvania and Michigan, district or 
school leadership in six of eight schools in 
the former state and six of nine schools in 



the latter state pushed their schools to focus 
on the state’s goals and standards, which 
teachers and administrators in some schools 
indicated was a response to public reporting 
and the competitiveness with regional 
schools. This overall influence of states in 
shaping school goals and potentially 
influencing the work of schools is 
particularly interesting considering that state 
departments of education have historically 
failed to influence the instructional program 
of schools due to their weak position relative 
to local agents and their role as 
administrators of federal policy and funding 
(Timar, 1997). 

Although we found that the states’ 
standards and assessment targets contributed 
significantly to the goals articulated by 
respondents in the schools, as stated earlier, 
respondents also discussed other objectives 
including the advancement of students’ 
social and emotional health, higher 
graduation rates and college attendance 
rates, and mastery of course content and 
skills both covered and not covered by the 
state’s assessments. 



Table 1 . Number of Schools in the Sample That Incorporated State 
Standard and/or Assessment Targets in Their Local Goals 



State 


Number of schools that 
incorporated state standards 
or assessment targets in goals 


State stakes 


California 


5 of 8 


High 


Florida 


7 of 8 


High 


Michigan 


6 a of 9 


Low 


New York 


5 of 7 


High 


North Carolina 


8 of 8 


High 


Pennsylvania 


6 of 8 


Low 


Total 


37 of 48 


— 



Tn three of these six schools, the state accountability goals were only incorporated into the math or the 
English department, not both departments. In all other cases, respondents reported on the role of state 
accountability standards and assessments across both the math and English departments. 
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Accordingly, the challenges and problems 
addressed by school respondents not only 
refer to issues that interfere with the 
school’s ability to improve student 
performance in mastering standards and 
with scoring better on state assessments, but 
challenges to broader academic 
development, student attainment, and 
student social and emotional development 
also appeared in our collection of responses. 

The challenges described by our 
respondents fell into two main categories: 

(a) problems embedded within the school 
and (b) problems embedded within students. 
The problems embedded within the school 
are those that are most within the control of 
the teachers and/or administrators. The 
problems embedded within students are 
those that from the perspective of teachers 
and administrators are carried into the 
school by the students and, therefore, 
problems that high school teachers and 
administrators can only ameliorate rather 
than prevent. The discussions we had in our 
sample schools revealed that by setting 
performance targets linked to a specific set 
of curriculum standards, accountability has 
highlighted problems (some new and some 
old) they are now compelled to address. 
Most of these new concerns appear among 
the problems embedded in schools. 

However, it can be argued that these policies 
have also highlighted and created a greater 
sense of urgency around some of the 
challenges that have been persistent in high 
schools over time and not directly linked to 
new requirements of the state policy. In the 
following discussion we describe the 
challenges teachers and administrators 
raised in our interviews. This description 
reveals not only the range of challenges 
teachers feel they contend with but also how 
the policy’s success in focusing attention on 
performance targets and state curriculum 
standards has influenced the matters that 
surfaced most frequently for our 
respondents. 



Problems Embedded 
Within the School 

We begin our discussion with problems 
embedded within schools because these 
were the issues that seemed most strongly 
influenced by the state policy. Issues related 
to curriculum and instruction as well as 
many of the concerns over school 
organization relate to the testing and 
curriculum requirements written into the 
state policies. In addition, several of the 
concerns related to teachers’ skill and 
commitment specifically refer to the new 
standards and expectations. 

Issues of Curriculum and 
Instruction 

As observers of accountability policies 
predicted, many schools found that the new 
standards and assessments required 
significant changes to what they taught and 
how they taught. Both administrators and 
teachers remarked on the weaknesses in 
their curriculum and teachers’ ability to 
teach the new curriculum. Respondents in 
1 1 schools described impediments related to 
curriculum and instruction and often drew 
the connection between these weaknesses 
and their state’s standards or assessments. 
Five of those 1 1 schools specifically 
indicated that some part of their curriculum 
or their entire curriculum did not align with 
the standards set by their state. Others 
attributed poor test performance by a 
subgroup of students or in a specific subject 
area to inadequate programs or programs 
that matched poorly with the assessments. 
The only curriculum-related concern raised 
by our respondents that did not directly 
relate to the state accountability system was 
an interest in providing a more rigorous 
curriculum schoolwide. 
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Table 2. Schools Reporting Curriculum and Instruction Problems 



State 


Number of 
schools with 
curriculum 
concerns 
related to 
standards and 
assessment 


Number of 
schools with 
curriculum 
concerns 
related to 
overall 
academic 
rigor 


Level 

of 

stakes 


Level of 
specificity of 
standards 


First full school 
year of standards 
implementation 
after legislative act 
or major policy 
revision 


California 


1 


0 


High 


High 


1999-2000 b 


Florida 


0 


2 


High 


High 


1998— 1999 c 


Michigan 


3 


r 


Low 


Low 


unclear d 


New York 


2 


0 


High 


High 


1999-2000 e 


North Carolina 


0 


0 


High 


High 


1996—1997* 


Pennsylvania 


3 


0 


Low 


Low 


1999-2000 g 



“This school is also one of the three Michigan schools that reported concerns related to standards 



and assessment. 

b As reported by the California Department of Education Web site. 
c As reported by members of the Florida Department of Education. 

d Due to several revisions in the state standards, the time at which the most significant change 
took place is unclear. 

e As reported by the New York State Education Department. 
f As reported by the Public Schools of North Carolina Web site. 
s As reported by Goertz and Duffy (2000). 



While it was notable that most 
curriculum issues raised by our school 
respondents could be linked to their state 
accountability systems, it is also interesting 
to note where curriculum issues did not 
arise. The combination of stability, maturity, 
and specificity of the state accountability 
policy seems to predict the extent to which 
schools continue to wrestle with curriculum. 
North Carolina was the only state in which 
the teachers and administrators did not 
discuss any challenges related to curriculum. 
Only two schools in Florida raised 
curriculum related issues, neither of which 
referred to their accountability system. 
Instead, both schools discussed the 
challenge of providing a rigorous academic 
program. Table 2 shows that of the six states 
in our study, North Carolina’s standards and 



assessment program has been in place the 
longest and ranks among the highest in the 
specificity of its standards and overall 
stability. Florida’s system, despite starting a 
few years after North Carolina’s, is also a 
relatively mature and stable policy with very 
specific standards. In contrast, schools in 
both Michigan and Pennsylvania with weak 
systems and low standards specificity seem 
to be still working out the curricular 
alignment and change expected under these 
accountability systems. 

School Organizational Issues 

Teachers raised a variety of issues 
related to the structure or operation of the 
school organization. The range of issues 
mentioned by teachers and administrators 
include the following: 
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• inadequate resources, 

• not enough time to cover the 
required curriculum, 

• excessive time requirements of 
standardized testing, 

• no time for teachers to meet or poor 
teacher collaboration, 

• school safety and discipline, 

• correctly categorizing English 
language learner (ELL) students, 

• large class size, 

• organization of school day that is not 
adequate — a block schedule is 
needed, and 

• rapid growth in student population. 

Of the various organizational issues, 
respondents most often identified resources 
as a challenge to improving students’ 
performance. In all, respondents in 22 
schools identified resource needs. The 
second most common issue related to the 
organization was related to time allocation. 
Issues related to time also dominated the 
organizational challenges. Teachers in seven 
schools found it difficult to deliver the 
required curriculum in the time allocated for 
their courses. Finally, teachers and 
administrators in nine schools raised issues 
of safety and discipline. All other 
organizational challenges listed in Table 3 
represent issues specific to only one school 
in our study. 

Teachers rarely drew the connection 
between organizational problems and the 
specific goals affected by these problems. 
These identified challenges most likely 



related to the tacit goal of operating a well- 
functioning organization characterized by a 
safe and orderly environment, by resources 
to provide teachers and students with 
adequate instructional materials and 
instructional time, and by competency in the 
important services provided by schools. 

Issues Related to Teachers 

The issues related to teachers fall into 
two constructs: (a) weaknesses in teachers’ 
skills and (b) unhelpful attitudinal positions 
such as resistance to change, low 
expectations, or overall weak commitment 
to their work or students. The range of 
concerns related to teachers’ skills include 
these: 

• teachers’ skills in general, 

• teachers’ ability to teach ELL 
students, 

• teachers’ ability to teach reading and 
writing, 

• teachers’ ability to teach in the block 
schedule, 

• new or out-of-field teachers, 

• limited opportunity to improve skills 
and perfect courses, and 

• ensuring that teachers teach what 
they report in lesson plans. 

The range of concerns related to teachers’ 
attitudes include these: 

• teachers’ resistance to change in 
curriculum. 
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• teachers’ resistance to change in 
practice, 

• teachers’ resistance to professional 
development (especially among 
veteran teachers), 

• teachers’ resistance to reading 
initiatives, 

• resistance of vocational teachers on 
academic performance, 

• teacher resistance to technology, 

• low teacher expectations for 
students’ success, and 

• teacher commitment. 

The identification of teacher-related 
challenges occurred in all states, with no 
state notably reporting more or fewer 
challenges in this area than the other states. 
Respondents in 17 of 48 schools identified 
weaknesses in teachers’ skills. Respondents 
that included a mix of teachers and 
administrators in 15 schools indicated that 
teachers in their school displayed resistance, 
lack of commitment, or low expectations, 
with teacher resistance to curricular changes 
and standards accounting for the concerns in 
10 of these 15 schools. This list made it 
clear that the requirement to change 
curriculum to match standards has in many 
places hit a sensitive spot with teachers. 

The instances of teacher resistance discussed 
in schools show teachers’ resistance to 
adopting the changes and training necessary 
to meet the demands of the state 
accountability curriculum as understood by 
people in the schools. 



Problems Embedded in the 
Students 

Lack of Student Motivation 

Concern over poor student motivation 
ranks as the most widely cited problem 
across respondents and schools in our 
sample and is not an uncommon issue 
among high school teachers (Siskin, 2003). 
Respondents in 41 of the 48 schools we 
visited explicitly remarked on the difficulty 
of working with students who lacked 
sufficient motivation to participate 
effectively in their education. In addition to 
general disinterest in class, respondents 
indicated that the lack of student motivation 
was manifested by poor attendance, poor 
preparation for class, or lack of effort on 
state assessments. This timeless issue should 
not be considered trivial, as many recognize 
that the sanctions and rewards often directed 
at adults in schools depend on the 
performance of students. Teachers directly 
mentioned the impact of students’ attitudes 
on their school’s performance outcomes and 
often viewed the students’ lack of interest in 
classroom work, test preparation, and the 
test itself as having direct consequences on 
their professional experiences. 

Weaknesses in Student 
Background 

Student background, which was linked 
to student motivation, represents another 
concern respondents felt affected them 
greatly but over which they felt only limited 
control. Respondents in 23 of the 48 schools 
remarked, in vague terms, that the students’ 
economic background, lack of parental 
support for education, or value structures 
that did not prioritize education impeded 
their efforts with students. In addition to 
comments on the students’ economic 
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background, respondents also raised the 
issue of students’ language background. 
Respondents in 10 schools indicated that 
students’ language proficiency was a serious 
issue with which they regularly contended. 
This issue posed a somewhat different 
challenge qualitatively than the background 
issues described above in that teachers 
recognized their own lack of training and 
skills to work with ELL students. 

Deficits in Students' Skills 

Responses from the teachers and 
administrators in our sample made the point 
that high school teachers faced an uphill 
battle to get their students to proficient 
levels in time for the state assessments or 
simply to perform high-school-level work. 
Respondents in 41 schools described the 
difficulty they faced in offering a high 
school curriculum to students who displayed 
significant deficits in fundamental skills or 
with classrooms of students who exhibited a 
broad range of skills. Our high school 
respondents also did not shy away from 
attributing theses deficits to their feeder 
school programs. Teachers in 30 schools 
explicitly commented on the poor 
preparation of students entering the school 
or the failed efforts to teach students 
fundamental reading and math skills in 
elementary and middle school. Interestingly, 
the schools expressing concern over the 
preparation of students spread over all of the 
states and, therefore, did not appear to be 
associated with level of stakes or the 
structure of testing (end-of-course exams 
versus one general assessment). 

Of the skills identified specifically, 
respondents most often cited their students’ 
deficits in the components of literacy — 
reading and writing. Teachers in 20 schools 
specifically discussed their students’ 
challenges with literacy. While the emphasis 



on literacy may have been an artifact of our 
interview strategy, which focused on the 
English and math departments, it may also 
reflect some important tensions between 
what high school teachers are “supposed to 
do” and what they are now expected to do. 
Teachers in several high schools we visited 
felt ill-equipped to teach adolescents 
reading, but four of the six states in our 
study assessed their high school students 
with a general reading comprehension 
exam. 10 It is important to note that teachers 
across all six states in our sample described 
the challenge of weak literacy skills in 
students. Although concern over deficits in 
math preparation did not receive the 
schoolwide attention of reading or writing 
skills, math teachers also raised the issue of 
student preparation. This issue was of 
particular concern in the districts and states 
that required all students to pass Algebra I. 

Summarizing Challenges 

Data on the reported challenges and 
impediments revealed that, for the most part, 
schools across all six of these states faced 
very similar issues and identified a broad 
range of challenges. (See Table 3 for a 
summary of the challenges raised by school 
respondents.) The most direct link between 
the accountability requirements and the 
challenges articulated by schools can be 
seen in the concerns over schools’ 
curriculum. However, it can be argued that 
the state policies have led to instances of 
teacher resistance as schools change their 
curriculum to match standards and meet the 
assessment expectations and that the state 
policies have intensified schools’ concern 



10 Pennsylvania, Florida, California, and Michigan 
assessed high school students with at test of reading 
comprehension and/or writing. New York and North 
Carolina required students to take assessments related 
to specific courses. They did not assess reading 
comprehension explicitly. 
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. Table 3. Summary of Challenges 



Challenges Share of schools in sample 

identifying the challenge 



Problems embedded within students 

Student skill deficits 


85% 


Low student motivation 


85% 


Student background 


48% 


Problems embedded within the school 

Weaknesses in curriculum 


27% 


Problematic teacher attitudes 


31% 


Weaknesses in teachers’ skill 


35% 


Problems with school organization 


71% 



over student skill deficits and motivation. It 
is important to remember that accountability 
must compel action as well as concern. Not 
all of the problems and challenges identified 
actually make it to the table. What ends up 
in decision situations is the topic of the next 
section 

Linking Solutions to 
Problems in Recent School 
Decisions 

The question of what problems were 
acted upon in decision situations completes 
the story of how accountability shaped the 
process of decision making. To what extent 
did accountability policy seem to influence 
an improvement response from schools, and 
to what extent did the policy shape what 
schools focused on in their improvement 
efforts? In this section we look at the new 
policies and programs described in the 
schools we visited in order to give the reader 
a sense of the issues that people in schools 
carried into decision situations. As we stated 
in the discussion of participants in decision 
making, individual teachers by themselves 
made a substantial share of the 



improvements targeting student performance 
in their daily efforts to confront challenges 
in their classrooms. They addressed these 
problems with the tools they controlled, 
including materials, instructional practices, 
the content of lessons, and their own time. 
Individual teacher decisions are critical to 
the schools we visited; however, the scope 
of an individual teacher’s influence is 
constrained. Policies and programs with the 
most substantial scope, attention, and 
resources are those made at higher levels of 
the organization such as the department or 
school level. Therefore, we only focus on 
the larger policies and programs adopted at 
the department and school level in this 
section. 

To get a full sense of the programs 
pursued by the schools, we concluded our 
interviews by asking for a comprehensive 
list of strategies employed by the school to 
address matters of student achievement. In 
an effort to get a sense of the academic and 
instructional problems that were addressed 
(by means other than individual teachers’ 
classroom efforts) we matched the strategies 
discussed by respondents at the school with 
the challenges identified by respondents in 
the school. For example, we matched a 
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school’s initiative to promote face-to-face 
conferences with parents programs with 
their articulated challenge of low parent 
involvement and challenges with student 
background. We matched a program to 
provide incentives for student test taking to 
the articulated challenge of low student 
motivation. Note that the matching reflects 
our evaluation of the strategy discussed and 
the challenges identified. The individuals at 
the school may not necessarily have 
matched the strategies with the challenges as 
we did, but nonetheless we feel that this is a 
reasonable approach to seeing what issues 
schools addressed. 

Overall, we found that schools tended to 
focus most significantly on issues related to 
student skills and curriculum, two 
challenges with relatively clear linkages to 
the state accountability system. Table 4 
shows that 85% of schools that identified 
student deficits as a problem described some 
program to address student deficits. 



Similarly, 69% of schools that identified 
curriculum weaknesses as a problem 
described their efforts to address the 
curriculum issues. Certainly, it may be 
argued that it is simple to add a remedial 
program like after-school tutoring in an 
attempt to address skill deficits and that is 
why such a high share of schools showed 
programs to address skill deficits. However, 
our data suggest that schools did not always 
take the easy way out in designing remedial 
programs, a point that will be addressed with 
more detail in the following sections. 
Furthermore, easy ways to address a 
problem would not necessarily account for 
the high share of schools addressing 
curriculum weaknesses. Not only does 
standards alignment take time and energy on 
the part of teachers, we also learned that 
several administrations and other school 
leaders faced significant challenges to the 
process of standards alignment from 
members of their staff. 



Table 4. The Challenges Identified and the Challenges Addressed 



Challenges 


Share of schools in sample 
identifying the challenge 


Share of those schools that 
identified the challenge that also 
had a program to address the 
challenge 


Problems embedded within students 

Student skill deficits 85% 


85% 


Low student motivation 


85% 


59% 


Student background 


48% 


17% 


Problems embedded within the school 

Weaknesses in curriculum 27% 


69% 


Weaknesses in teacher skill 
and commitment 


58% 


36% 


Problems with school 
structure 


71% 


24% 
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While it was interesting to see the 
response to skill and curriculum weaknesses 
in the high schools, it is also interesting to 
take note of those problems that schools 
address the least. We might expect that those 
problems positioned outside the school 
would be considered the most difficult for 
schools to confront and, therefore, least 
often addressed. As we can see from Table 
4, schools that identified students’ 
backgrounds as a problem rarely 
implemented programs to confront this 
problem. However, a surprising share of 
schools seemed to take on the issue of 
student motivation, with more than half of 
the schools that identified motivation as a 
problem implementing some program to 
engage students. Although the skills and 
commitment of teachers seem like problems 
within the control of those in the school, 
only 10 of the 28 schools in which 
respondents articulated problems with 
teachers’ skills or commitment described 
any schoolwide or departmental effort to 
address their concerns. Importantly, only 
one of the schools in which respondents 
discussed the problem of teacher resistance 
had any effort to address this issue. In this 
one school the principal was pursuing a 
“reconstitution by attrition” in which she 
welcomed retirements and unhappy transfers 
as an opportunity to recruit like-minded 
teachers. 

Through this analysis, we learned that 
schools overall addressed a wide range of 
strategies, but not surprisingly, they did not 
address all the problems identified by the 
respondents. The fact that schools did not 
address all the problems they identified for 
us likely reflects the fact that the schools 
could not address everything at once. 
However, this result also reflects the fact 
that our interviews sought information on 
efforts to improve student performance. We 
simply did not pursue at much length 
schools’ efforts to deal with some of the 



more procedural or administrative problems 
discussed by respondents. In addition, 
schools appeared to select strategies that did 
not necessarily correspond with the 
problems identified by the respondents. A 
variety of explanations possibly explain why 
schools might be implementing strategies 
for unarticulated problems. Some problems 
such as low student scores may be so 
fundamental to the school and its goals that 
teachers did not specifically articulate them. 
In addition, agents external to the school 
such as districts, external partners, or state 
departments of education may require 
programs that may or may not correspond 
with the problems identified by local 
teachers. 

Decision Situations and 
Their Participants 

The nature of decision situations gives 
some indication of the formality of the 
decision process used to address the 
problems outlined above, while the range of 
participants illustrates the different levels of 
decision makers at work in schools. 
Although the success of the policy relies on 
good decision making by local schools, the 
states offer little guidance or support to 
schools that would encourage or facilitate 
formal or coordinated schoolwide needs 
assessment and decision-making processes. 
Only two of the states we visited, California 
and North Carolina, included a system for 
supporting school improvement decision 
making in the body of the policy. Both of 
these states introduce external participants in 
the decision-making scene. North Carolina 
required its lowest performing schools to 
work with a state-designated team for school 
improvement, and California required 
schools participating in the Immediate 
Intervention/Underperforming Schools 
Program (II/USP) to engage with an external 
partner to evaluate the school’s need. 
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Because these states only offered this 
support to its lowest performing schools, 
these efforts were not widely represented in 
our sample, and we see no evidence that the 
support offered to the states’ lowest 
performing schools served as a model for 
other schools attempting improvement. 

The picture of decision situations 
related to improving student achievement 11 
and their participants in the 48 schools 
showed a blend of formal and informal 
decision situations with informal and ad hoc 
efforts by individual teachers, 
administrators, or other groups in the school 
occurring most often. As might be expected 
with informal decision making, the decision 
process did not always flow, as logic would 
predict, with the identification of need 
preceding the selection of strategies for 
improvement, and the participants in 
decision making were often not organized in 
accordance with the organizational structure 
of the school. Decision situations in the 
schools we visited seemed to arise from 
three scenarios: (a) a problem was 
recognized, (b) a solution was identified, or 
(c) a cycle required a decision. We found 
several decision situations that formed 
because individuals or groups of individuals 
identified a problem in the school that they 
wanted to address. For example, a math 
department in a school had been struggling 
with inappropriate placement of students in 
classes. They decided to use their 
department meeting time to discuss and 
resolve this issue. These types of decision 
situations took advantage of existing 
structures such as departmental meetings or 
planning team meetings but also led to 



1 1 While we tried to focus discussion on decisions 
intended to improve student achievement, our queries 
into the most recent departmental decisions or recent 
schoolwide decisions often yielded information on 
decisions that seemed unrelated to improving student 
achievement. For example, members of one 
department described a recent decision to move the 
refrigerator in the faculty lounge. 



decision situations using less formal or ad 
hoc arrangements of people. 

While the practice of identifying a need 
then creating an opportunity to resolve the 
need appeals to logic, we learned about a 
number of decisions in which the identified 
need seemed to follow the identification of a 
solution. In these cases a solution seemed to 
catch someone’s attention and this solution 
was later matched to a problem, common 
practice that appears in Cohen, March, and 
Olsen’s (1972) description of “garbage can 
decision making.” These authors explain 
that this practice often occurs when decision 
makers have unclear preferences, unclear 
technology, and continuously changing 
participants in the decision situation, 
however, the cases we observed are more 
aptly described as decision makers with 
unclear needs. In most cases where solutions 
led decision situations, the individuals in the 
decision situation had never made a 
thorough evaluation of their needs or the 
types of strategies that may ameliorate this 
need. In one illustrative case, an assistant 
principal learned about the “middle school” 
model, which offers a school design that 
addresses the developmental needs of 
students in the middle school years. He 
thought this model would be appropriate in 
his school, which housed grades six through 
twelve. He raised the idea with his principal, 
and together they decided that this program 
would help their younger students to 
transition to the high school. From our 
interviews with the principal and assistant 
principal it was clear that they had not 
discussed a problem with the younger grades 
prior to this proposition, but nonetheless 
they identified some problems that matched 
the identified strategy. In this case and with 
similar cases in other schools, the desire to 
implement these specific solutions drove the 
entire decision process. 

Respondents in a handful of schools 
mentioned their engagement in cyclical or 
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required decision situations. Of the three 
decision-making prompts discussed in this 
section, these events made most consistent 
use of formal organizational forms and 
appeared the most structured. Interestingly, 
few schools mentioned the regular 
development of school improvement plans, 
one of the most common recurrent decision 
situations in schools because districts and 
states often require the annual development 
of plans (O’Day, 1999). State accreditation 
and evaluation processes, which recurred on 
regular intervals, required four of those 
schools (three in Michigan and one in 
Pennsylvania) to convene a team and 
develop a school plan. Two schools 
discussed their district’s curriculum renewal 
cycle in which each subject area, in its own 
turn, engaged in evaluation, revision, and 
implementation phases. Finally, three 
schools in our California sample participated 
in the II/USP program and were required to 
develop an assessment and improvement 
plan with the help of an external consultant. 
This requirement led to decision situations 
related to the development of a school action 
plan. Interestingly, these were the only 
decision situations that directly resulted 
from the state accountability policy. The 
role of II/USP and work of external 
consultants in California’s schools will be 
discussed in more detail in later sections. 

Given the both formal and ad hoc 
decision events in the schools we visited, it 
should not be surprising to learn that the 
participants in the decision situations ranged 
widely. The decision-making groups 
included formally specified departments and 
planning teams. However, consistent with 
the large share of informal decision 
situations, but perhaps inconsistent with 
prior research on the importance of 
departments in high schools, we found a 
relatively weak effort to exploit these 
organizational structures. Instead, we found 
that individuals and self-formed groups — 



groups not designated by an organizational 
structure — were the most common 
participants and participant groups in 
decision making. As per the fundamental 
principle of limited mandates inherent in 
outcomes accountability, states took little 
direct role in determining participants in 
decision situations. States generally did not 
require that schools demonstrate schoolwide 
participation in decisions and did not require 
schools to involve their districts in 
decisions. As stated above, only California’s 
and North Carolina’s policies required 
certain participants be brought to the table, 
but this was only required of their lowest 
performing schools. Our data, however, 
show that state accountability may have 
indirectly influenced the participants in 
school decisions by prompting more 
involvement from their districts. While 
most of our schools had multiple types of 
decision makers and decision-making 
groups working with different issues, certain 
groups clearly dominated the scene in some 
of the schools we visited. Overall, we found 
a surprisingly strong role by districts and a 
relatively weak effort to exploit 
organizational structures such as 
departments and schoolwide planning teams. 

Although the state accountability policy 
does not attempt to play a strong role in 
decision making, it is important to 
understand the range of decision makers in 
schools and the prevalence of each for 
decisions. Different types of decision 
makers have different levels of authority and 
influence in schools, and the scope of issues 
addressed in schools depends greatly on the 
types of participants attending to decision 
situations. For example, individual teachers, 
though they have significant impact on 
students, have only limited scope of 
influence. If one teacher changes her 
materials, only her classroom will be 
affected. However, if the department 
collectively makes a decision to change 
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materials, all students taking courses in the 
department will be affected. Below we 
outline the different participants in decision 
making including individuals , groups, and 
external agents in decision making in the 
high schools we visited. 

Independent/lndividual Decision 
Makers 

Individual teachers and administrators 
making decisions independently represent 
the most consistent decision makers across 
the high schools we visited. The norm of 
teacher autonomy, described in the theoretic 
section above, was evident in the regularity 
with which teachers took independent 
initiative to make adjustments in their 
practice, curriculum, or materials. In every 
school we visited, we heard testimonials of 
teachers seeking solutions to daily concerns 
about students or lessons as well as broad 
concerns regarding practice, assessment, 
curricular approaches, and student academic 
performance. However, the domain of a 
teacher’s decisions only extended to the 
teacher’s classroom and, at times, only 
affected individual students. His or her 
individual initiative, while significant and at 
times constituting the major improvement 
efforts being made in a school, does not 
imply a school reform effort. 

Principals in our sample described 
making decisions regarding schoolwide 
improvement strategies independently of the 
faculty or even other administrative staff. 
Unlike teachers, principals’ independent 
decisions reflected school-level decisions 
and they generally affected many 
individuals across the organization and 
touched the instructional work of teachers. 

In our sample, principals took this 
centralized approach because this approach 
aligned with the historical role of 
administrators in the school or in order to 
wrest control of a school with substantial 



needs. The authority vested in principals 
was particularly clear in our sample, where 
teachers in 13 of 48 schools described 
significant instructional improvement 
decisions in which the principal did not 
involve the staff in the decision-making 
process. 12 

Decision-Making Groups 

Departmental decision makers. The 
department structure, which is one of the 
most widespread organizational forms in 
high schools, offers a logical decision- 
making body for schools. Members of 
subject-based departments share common 
academic expertise, participate in common 
courses or a sequence of courses, share 
many instructional practices, and, 
importantly, share students (a commonality 
even in interdepartmental teams). During 
our field study, we focused heavily on the 
role of English and math departments in the 
school improvement process because 
previous research on high schools has 
emphasized the importance of departments 
in teachers’ professional lives. In each of the 
schools we visited, we specifically queried 
teachers on the decision-making role of 
subject-based departments and other teacher 
groups that exist in the school. All but two 
of the schools we visited recognized subject- 
based departments. Of the two schools that 
did not recognize subject based departments, 
one recognized interdepartmental teacher 
teams, and the other recognized no teacher 
teams. 13 Schools supported departmental 
and team structures by formally recognizing 



12 It is interesting to note that the centralized decision 
making was not necessarily contested by teachers in 
these schools. Most of the teachers who commented 
on the centralization of decision making seemed to 
accept this arrangement as the role of the principal 
and the norm for the school. 

13 Despite having no formal departments, these 
teachers continued to identify professionally and 
informally with their subject-level colleagues. 
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the groups with designated leaders 
(department chairs/team leaders), 
responsibilities, requirements to meet as a 
group, and, in many cases, supply budgets. 
Departments or teams held meetings in 
nearly every school we visited. However, 
the frequency and regularity of those 
meetings varied significantly across schools. 
Teachers in most departments described 
meeting infrequently (once a month or less) 
or meeting on an “as needed” basis, while 
some met as frequently as once a week. 
Overall, the departments in this study did 
not come across as strong decision-making 
units. 

Looking across our sample, we rarely 
discerned a specific role expected of the 
departments. Informational interaction 
dominated as the style of interaction for 
department members. Administrations rare 
distributed decision-making authority to 
their departments, and teachers did not 
describe their departments as key decision 
makers in their schools. On the whole, 
departments served as a central decision- 
making authority in their schools most 
frequently in California (n = 4) and New 
York ( n = 3). In Florida and especially 
North Carolina, among the strongest 
accountability states in our sample, teachers 
in only 1 of the sixteen schools identified 
departments as key decision-making units in 
the school. Instead, teachers indicated that 
the principals and/or central offices wielded 
substantial authority, and any distributed 
authority resided at the teacher level. In 
addition, teachers in 30 of the 48 schools did 
not report on any significant school 
improvement strategies selected, in part or 
entirely, by either their English or math 
department. Among the 18 schools where 
teachers reported that departments made at 
least some important decisions, we found 
considerable variation in the types of 
decisions and the regularity with which the 
departments were brought into the decision 



process. While we cannot conclude that 
accountability policies or the level of stakes 
impeded the use of organizational structures 
such as departments for decision making, it 
is clear that the press of these policies did 
not compel schools to take advantage of 
these forms to help decision making. 

The conclusion that departments played 
a relatively limited role in decision making, 
however, seems to contradict a substantial 
body of work demonstrating the importance 
of departments in schools. McLaughlin and 
Talbert (McLaughlin & Talbert, 2001) and 
Siskin (Siskin, 2003) described departments 
as the center of professional life for teachers 
and an important source of intellectual, 
professional, and social development for 
teachers. We do not deny that the 
departments in most of the schools we 
visited played a vital role in the social and 
professional lives of teachers. As 
McLaughlin & Talbert and Siskin have 
reported, the departments provided teachers 
with informed colleagues who dealt with 
similar topics and student issues to which 
they could turn for ideas. However, 
department meetings typically provided a 
forum for conversation from which 
individual teachers would acquire 
information for their independent decision 
situations or learn about administrative 
issues such as new or proposed school and 
district policies or time and dates for 
professional development opportunities. We 
saw only rare cases in which administrations 
or districts mobilized their departments to 
make decisions about school improvement. 
By and large, the departments served as 
salient social and professional units for 
teachers but they did not serve as important 
units for organizational decision making. 

Schoolwide committees or faculty 
senates. Policy designers often tout the 
advantages of schoolwide committees in 
decision making, citing the benefits of using 
the broader base of knowledge and 
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promoting teacher buy-in for the strategies 
selected. In the schools we visited, 
requirements to produce school 
improvement plans as well as state 
accreditation and evaluation procedures 
prompted schools to create schoolwide 
committees for needs analysis and planning. 
In other cases, standing committees, such as 
committees of department chairs or 
occasional committees convened to deal 
with specific issues, served as forums for 
discussing and selecting programs or 
addressing concerns. Despite the purported 
advantages of schoolwide committees, only 
13 of the 48 schools we visited discussed 
using schoolwide teams as decision-making 
bodies, and teachers in one of these schools 
explicitly stated that the team did not do 
much for the school. 

Participants External to the School 

Countless educational providers 
including districts, universities, and private 
educational consultants stand at the ready to 
assist high schools in their improvement 
efforts. Several schools in our sample drew 
from districts, schoolwide model providers, 
and hired evaluators or school planning 
specialists in making instructional 
improvement decisions. 

Districts. The district (represented by 
curriculum coordinators, directors of school 
improvement, and directors of secondary 
education) were, by far, the most influential 
external agents in the schools we visited. 
Teachers and administrators in 18 of the 
schools we visited reported that their district 
selected significant strategies that the school 
was either required or strongly encouraged 
to use. In four more schools, members of the 
district central office worked collaboratively 
with administrators, leadership teams, or, in 
one district, departments to help strategies 
for improvement. Literature on decision 
making and the application of information in 



organizations recognizes that power 
structures play important roles in 
determining the outcomes of decision 
situations (Brown, 1993). The districts in 
our sample enjoyed the benefit of authority 
they received as knowledgeable, trusted, and 
respected information providers as well as 
the benefit of power bestowed by the 
hierarchical nature of the educational 
institution. With this authority districts 
selected and implemented improvement 
strategies in the schools we visited. 

Although we saw several instances in which 
schools appreciated the suggestions and 
support of their districts, we also saw 
instances in which schools grudgingly 
implemented the district’s programs or even 
resisted, intentionally failed to implement, 
or weakly implemented the district’s 
strategy. 

As stated earlier, a link between district 
involvement and the design of 
accountability policies seems likely but is 
still unclear from these data. Only 
Pennsylvania’s accountability system made 
districts their primary target for sanctions, 14 
but even Pennsylvania exercised its 
authority to sanction districts in relatively 
few cases. 15 Though the state has exercised 
its authority sparingly, the superintendent 
from one district in our sample indicated 
that he feared the sanctions that other 
districts in the state had received. Though no 
other district administrators directly stated 
that their intervention was a result of the 



14 California and Michigan had each taken over 
districts in their states, but these takeovers were in 
response to extreme cases or were a result of fiscal 
crisis in districts and not linked to the state’s current 
accountability policy. 

15 At the time of data collection, Pennsylvania had 
listed 10 districts on its “empowerment list,” which 
makes the district eligible for state intervention. The 
state had only identified two districts as 
“empowerment districts,” which indicates that the 
state could exercise even more authority than with 
the schools on the list. 
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state’s accountability pressure, six of the 
eight schools in our Pennsylvania sample 
reported a strong district role. In the rest of 
our data, however, the involvement of the 
district did not seem to be related to the 
strength of the state’s accountability system. 
Five of the eight schools we visited in North 
Carolina, a high-accountability state, 
reported strong district roles, while no more 
than three schools in New York and 
California, the other two strong 
accountability states, reported the district 
making important programmatic decisions 
for the school. This result suggested that the 
level of district involvement did not depend 
on the level of stakes created by the 
accountability system but on other factors at 
the district and school level. 16 

Status in the state’s accountability 
system seemed to play a role in determining 
which schools received special attention 
from the districts, but even this link is not 
perfect. Florida, North Carolina, and 
Pennsylvania each had one school in which 
the district took a strong role because the 
school had been classified as low- 
performing by the state or was among the 
districts’ lowest performing schools. 
However, a school’s accountability status 
did not necessarily imply strong district 
intervention. Our sample also included a 
handful of schools that were among their 
districts’ lowest performing or had received 
low marks in their states’ accountability 
system and that did not receive strong 
intervention. 

The role of districts in California 
showed the most direct link between district 
intervention and accountability. Districts in 
our sample actually seemed to co-opt the 
state’s policy mechanisms to intervene with 
their own low-performing schools. Our 
sample included three schools from two 



lh Weinbaum (in this publication) explores in more 
detail the conditions that may be leading to district 
involvement. 



different districts that had been identified as 
in need of improvement and participated in 
the state’s II/USP and, therefore, were 
required by the state to hire an external 
evaluator. In each case, the districts selected 
their schools’ external evaluators. Clearly, 
these schools received support from the 
district because they had been identified as 
low-performing and received support that 
was prescribed by the accountability policy. 
(More details on II/USP follow.) 

External providers and state policy: 

The case of California’s II/USP. The 
structure of the accountability system and a 
school’s experience with accountability, 
however, seemed to play a part in the extent 
to which schools accessed nondistrict forms 
of external assistance in making school 
plans and selecting strategies. Five schools 
described involvement of external assistors 
in their search for strategies. The most 
prominent use of external assistance across 
the states we visited was in California, 
where three schools in our sample 
participated in the state’s II/USP. The 
California case provides the most direct link 
between decision making and the state’s 
accountability policy and, therefore, merits a 
detailed discussion in this chapter. 

California identified each of these three 
schools as in need of improvement and, 
therefore, eligible for participation in the 
II/USP, which is voluntary. As participants 
in the program, schools were required by the 
state to hire an external evaluator from a 
state-approved list of assistors to plan the 
school’s improvement strategies, for which 
the state provides funding for 2 years. The 
intended impact of this program on decision 
making is to force schools to include 
evaluators and an assistance team in the 
decision-making process and to increase the 
quantity and quality of information used in 
the decision-making process. 

Although II/USP nominally identifies 
schools as the locus of decision making and 
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change, the selection of external evaluators 
becomes yet another point at which districts 
often exploit the power vested in them by 
the hierarchy of educational institutions. 
Districts often take the lead in identifying 
external evaluators and matching them with 
schools. As cases in point, both of the 
California districts with II/USP schools in 
our sample selected the participating schools 
and chose the external evaluators to work 
with those schools. Each district selected a 
single external evaluator to work with all of 
its II/USP high schools. Our sample 
included two schools in each district, but 
only one of the high schools in the first 
district participated in II/USP. In the second 
district, both of our sample schools 
participated in II/USP. 

Our sample revealed that the interaction 
between external evaluators and schools 
varied across schools. Although the state 
must approve the external evaluator, it was 
clear from comments made by our 
respondents that these providers offer 
different types of services to the schools that 
hire them. Just as local context and acts of 
resistance constrained districts’ efforts in 
schools, local conditions affected the impact 
of external evaluators. The school with the 
most successful II/USP experience 
demonstrated a collaborative arrangement 
with its external evaluator. While the school 
had already begun to search for solutions to 
the many problems of an urban high school 
with a large immigrant student population, 
assistance by the external evaluator 
appeared to direct the search toward a 
coherent set of solutions. Two factors 
seemed important in this successful II/USP 
experience: (a) the school’s strong 
leadership and communal culture and (b) the 
school’s strong commitment to the ongoing 
implementation and improvement of the 
action plan developed during the II/USP 
process, a condition also found in a recent 
comprehensive examination of II/USP by 



the American Institutes for Research (AIR) 
(O’Day & Bitter, 2003). 

Other external providers. Two 
additional schools, one in North Carolina 
and one in Florida, received external 
assistance in their search for improvement. 
However, the assistance received by each of 
these schools was very different. The school 
in North Carolina received attention from a 
local university that “adopted” the school. 
The university did not require any payment 
from the school or the district for their 
assistance. They collaborated with the 
district and school to locate resources and 
plan an improvement strategy for the school. 
The school in Florida, by adopting a 
schoolwide model, purchased a plan for 
school improvement. The school model 
included a prescribed program for 
curriculum and school organization. 
Although research on external partners 
suggests that schools maintain some control 
over the plan to be implemented (Finnigan, 
O’Day, & Wakelyn, 2002), the model 
essentially made a number of decisions 
about school change and what it would look 
like in this school. Interestingly, both of the 
schools using these external providers hailed 
from high-accountability states and had 
earned the lowest or next to lowest ranking 

17 

in their respective accountability systems. 
While the use of external assistance in only 
two schools certainly cannot confirm that 
high stakes compels schools to seek external 
assistance in planning school improvement, 
this may be an interesting question to pursue 
in larger samples. 



17 The two schools referred to in this statement came 
from Florida and North Carolina. The school from 
Florida was labeled a D school (the second to lowest 
ranking) by the state. The school from North Carolina 
was identified as a priority school. 
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Making the Decisions: 
Information and 
Solutions 

The discussion to this point has focused 
on the ways in which the states, through 
their application of goals and sanctions, 
have influenced the process of decision 
making in schools. However, both advocates 
and critics of accountability say that 
focusing goals and improvement efforts is 
not enough to ensure the success of the 
policy. These policies, which require that 
schools teach all students to the same 
standards in a growing number of academic 
subjects, represents a substantial shift in the 
purpose of high schools (Siskin, 2003), and, 
therefore, high schools must select and carry 
out strategies that produce a shift in 
instructional programs and practices that 
will match the shift inherent to this policy. 
Furthermore, critics of accountability argue 
that the changes in programs’ curriculum 
and practice must do more than strategically 
target students, narrow curriculum, and 
focus on test preparation skills, activities 
some researchers argue have happened 
under accountability conditions (McNeil, 
2000). In the discussion that follows we 
explore to what extent schools seek out and 
select strategies to represent the shift in 
practice Siskin (2003) argues may be 
necessary and avoid the strategic but only 
surface strategies of which McNeil warns. 

In this section, we take a look at the 
information that is used to select strategies 
for reform and a range of solutions chosen 
by high schools in their efforts to improve 
students’ performance. We frame this 
discussion around the model of search and 
selection proposed in the literature on 
bounded rationality, a model we earlier 
argued relates to the conditions of 
educational organizations in the context of 



accountability. Bounded rationality predicts 
(a) that schools will turn most frequently to 
ideas that are already held within the school 
or are very close to the school and (b) that 
schools will favor strategies that maintain 
their current practice by adding onto the 
core program or targeting students within 
the core program. This model, therefore, 
predicts what many accountability critics 
argue is the policy’s weakness — the 
incentive to engage in short-sighted 
strategies that are peripheral to the core 
technology of schools. 

Our examination of schools’ search and 
strategy selection confirmed this prediction 
in many ways. Schools by and large relied 
on local information and adopted strategies 
that “tweaked” their current program or 
dealt with issues by having splinter 
programs that did not disrupt the traditional 
program. However, we also found ways in 
which barriers to new information were 
overcome and situations in which schools 
pursued strategies that changed their core 
technology. We found that districts provided 
a dominant force leading to the use of new 
information and strategies. The sections that 
follow describe the ways in which schools 
revealed the predictions of bounded 
rationality. However, in these sections, we 
also provide an extensive discussion of how 
schools did not conform to the model’s 
predictions in an effort to illustrate for 
district leaders and policymakers the role 
districts can play in providing schools with 
new ideas and supporting reform. 

The Conditions for Finding 
New Ideas: The Search for 
Information in Schools 

Our data offer additional support to the 
existing literature supporting the basic 
premise of the bounded rationality theory. 
Teachers in some rural areas described their 
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professional development options as limited. 
Teachers across our sample often described 
their professional development opportunities 
as too general to be put to use. Several 
teachers discussed the impact their tight 
schedules had on their ability to go outside 
the school for information; meanwhile, few 
schools described coherent efforts to bring 
new ideas or information on curriculum or 
instructional practice to teachers. Principals 
also described their own time constraints 
and the budget constraints that restricted the 
opportunities that could be made available to 
teachers. Contributing to the constraints 
imposed by time and resources were the 
circumstances under which most decisions 
in the schools we visited were made. Recall 
from the sections on decision situations and 
participants that most of the decisions 
described for us were made by individuals 
or ad hoc groups that had no formal 
recognition by the school’s organizational 
structure. These individuals or groups rarely 
had resources backing their decision-making 
efforts. Furthermore, many of the decisions, 
even in the relatively rare cases in which 
departments were making decisions, were 
carrying out relatively informal and quick 
searches on an “as needed” basis. In many 
of these cases, decision makers attempted to 
address the problems relatively quickly with 
one or two meetings. 

Resources were not the only culprits in 
limiting access; our research team found that 
in several cases teachers and administrators 
showed a “benign neglect” of ideas outside 
the school. Teachers and administrators 
often showed almost no awareness that they 
could or should look beyond their 
experience or their colleagues for 
information on new strategies, and they 
showed little knowledge of how to do so. 
While this failure to consider outside 
information was not explicitly mentioned by 
the originators of bounded rationality, this 
behavior seems a logical consequence of 



working in contexts with what is perceived 
to be unclear, inconsistent, or unreliable 
information. While teachers’ perception that 
strategies were not available to be found was 
less obvious in our data than the constraints 
on access to information, teachers in one 
school commented explicitly on the lack of 
good programs on adolescent literacy. 

Given the constraints described above 
and as predicted by the model, teachers’ and 
administrators’ reliance on information, 
knowledge, and skills that teachers and 
administrators already possessed appeared 
overwhelming in our initial impressions of 
the schools we visited. Much as with 
Huberman’s (1983) conclusion that teachers 
generally view valid information as 
information received from local colleagues, 
we found that teachers seemed most 
confident with the resources they had at 
hand in their schools. Researchers visiting 
schools often remarked on the lack of search 
anywhere outside the school. Reports by 
researchers from the field on each school 
were filled with descriptions of teachers’ 
efforts to learn from their colleagues and to 
resolve issues by calling on their own 
experience or the experience of their 
colleagues, typically departmental 
colleagues. At the departmental level, 
respondents reported that they often arrived 
at decisions by pooling the information of 
the department members and drawing from 
the collective knowledge and experience of 
the departmental members. Teachers 
described their departments’ decision 
making as a process that could be 
characterized as “putting heads together” to 
come up with a strategy or running with an 
idea one member brought to the table. 
Schoolwide teams described efforts much 
like the departmental efforts to draw on the 
collective knowledge and skills of the team 
members. Finally, principals described using 
their own experiences to select strategies or 
programs for the school. 
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Organizations, however, rarely 
represent just one style of search and 
selection, and our sample of schools was no 
exception. While the vast majority of 
decision making in schools relied on internal 
information when making decisions at every 
level of the organization, the schools in our 
sample challenged the model’s predictions 
and revealed a variety of avenues through 
which information from outside the school 
entered the organization. Schools in our 
sample benefited from instances when 
information was actively brought to the 
schools as well as instances in which 
individuals inside the school sought out 
information. 

Interestingly, districts in our sample 
appeared to provide the most common 
avenue along which information about 
strategies or approaches reached into 
schools. This is possibly one of the most 
significant findings in this examination of 
the decision and search process of high 
schools. Respondents in more than half (26) 
of the schools in our sample explicitly 
commented that they learned about one or 
more of the improvement strategies they 
used in the school when their district either 
suggested or required the strategy. Districts 
in our sample introduced a wide variety of 
strategies to their schools, including new 
curriculum programs, new assessment tools, 
new remedial classes and/or curriculum, and 
new school schedules or organization. As 
explained in the earlier discussion of 
participants in decision making, the districts 
played an active and valuable role in the 
work of schools. With authority and 
resources behind them, districts used a 
variety of approaches to bring new ideas 
into schools. At times the district introduced 
a strategy by requiring all schools or schools 
in a subgroup such as low-performing 
schools to implement a specific strategy. For 
example, one district in our study required 
all of its schools to implement a package of 



grade and course make-up programs, a 
strategy that had not been used in the 
district’s high schools. When districts 
required all of their schools to implement a 
strategy, they typically had little discussion 
with school administrators or teachers about 
the design of the specific strategy. Districts 
also offered unsolicited suggestions of what 
schools could do. For example, several 
schools in Florida described developing an 
intensive reading course that targeted ninth- 
grade students who performed poorly on the 
eighth-grade assessment. Teachers and 
administrators in these schools explained 
that the idea of an intensive reading course 
came from their districts and was presented 
as a possibility for their school and others 
that struggled with weak readers. 

In several cases, schools actually 
solicited information from their districts and 
turned to their district administrators for 
help in resolving issues in their school or 
worked collaboratively with schools. An 
example of this collaboration was seen in a 
Florida district that assembled a team of 
content area specialists to work with each of 
the schools that the district identified as its 
greatest need schools. This team met 
regularly with the administration and 
members of the departments with which the 
members shared a specialty, and together 
they developed a plan for improvement and 
decided on strategies to address the school’s 
needs. Teachers and administrators in this 
school explained that the district’s team 
members often presented them with ideas 
that they would consider and often 
implement. 

Our sample also showed a situation in 
which the district planted administrative and 
consultant personnel into a school and 
charged these individuals with carrying out a 
specific school improvement plan designed 
by the district and based on new curriculum 
standards. In fact, at this school the principal 
admitted that he was brought into the school 
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because he was trusted by the district to 
implement the district’s plan, and because 
he planned to retire he had no need to make 
friends with the staff. He only aimed to 
implement the plan. While not common in 
our sample, this method of “grafting” 
(Huber, 1995, p 136) individuals with 
specific skills or ideas often appears in 
literature on organizational learning. 

Professional development by teachers 
represents one more means through which 
districts, as regular providers of professional 
development, brought information into 
schools, but it also represents an important 
means through which information from a 
variety of sources spanned the boundaries of 
the organization. This chapter emphasizes 
the importance of teachers’ daily decisions 
for understanding the scope of the efforts 
being made in schools. The discussion just 
above reported that teachers based the vast 
majority of their decisions on their own 
knowledge and skills or that of their 
colleagues. Many teachers, however, are 
continually updating their repertoire of 
knowledge and skills through regular 
participation in professional development on 
a variety of topics. Although teachers who 
have taken a professional development 
course on integrating writing in their classes 
may not immediate introduce new writing 
assignments, they may do so as the demand 
to improving reading and writing increases. 
Professional development serves as an 
important means through which teachers 
become exposed to new practices and 
curriculum. 

Teachers in this study described a wide 
range of professional development topics, 
with some of the most common including 
classroom management, test preparation, 
coping with weak readers, writing across the 
curriculum, understanding and adopting 
standards, and curriculum alignment. While 
professional development on classroom 
management has been a part of professional 



development, topics such as standards 
alignment as well as reading and writing 
across curriculum areas represent topics 
receiving new emphasis in light of state 
assessments on reading and wring. The one- 
shot or short series workshop format 
dominated the type of professional 
development described by the teachers in 
our study. However, teachers also described 
professional learning through 
communication with teachers outside of 
their school, independent research, or 
accessing the Internet. 18 Teachers’ own 
professional development and, because 
teachers drew heavily from each other, the 
professional development of their colleagues 
introduced variation into the existing 
knowledge and skills of the teaching staff. 
Although teachers rarely mentioned 
situations in which they sought professional 
development in response to a specific 
immediate concern, they mentioned that 
they had, at times, drawn from the 
information they received at prior 
professional development sessions and, no 
doubt, received the benefit of their 
colleagues’ professional growth when 
turning to their colleagues for assistance. 

Teacher professional development as 
pursued in most schools, however, is not the 
most efficient way to get information into 
schools, because teacher selectively 
obtained and retained the information they 
received through their professional 
development experiences. The professional 
development of teachers in our sample was 
largely teacher-driven and almost 
completely dependent on the initiative of the 
teachers. Consequently, the information 
introduced to teachers was highly 



18 The use of the Internet was mentioned enough that 
it may be worth an independent investigation that 
examines what teachers search for on the Internet, the 
quality of the materials they receive from the 
Internet, and how communities of teachers on the 
Web might influence how we think about teachers’ 
professional community. 
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unsystematic and varied widely with on the 
teachers’ own initiative, interests, 
assessment of their professional learning 
needs, and professional requirements. In 
addition, because teachers fulfilled an 
interest or a requirement with their 
professional development without 
necessarily identifying a need to update or 
change their instructional practice, what 
teachers retained was also highly unreliable. 
If the teacher finds the content of the 
workshop compelling enough, she might 
retain the information but a considerable 
amount of information is simply lost 
because the teacher has not found the 
workshop compelling, or the ideas conveyed 
are not met with support or follow-up after 
the workshop. 

Another significant means through 
which information entered the decision- 
making arena from outside the organization 
was through the principal. That is, the 
principal served as the boundary spanner for 
the organization. Thirteen principals in our 
sample described their efforts to attend 
conferences, attend workshops, read trade 
journals, and/or engage in principal 
networks intending to learn about new 
strategies or approaches that could be used 
in the school. Like the teacher professional 
development, this form of search had 
limitations. Principals generally engaged in 
these activities independently or shared the 
experiences with only closely situated 
assistant administrators. This information 
typically reached teachers only when the 
principal acted on this information to create 
school policy. Interestingly, most of the 
principals who described these independent 
search efforts were also principals who 
played dominant decision-making roles 
according to their schools’ respondents. 

Other sources of external information 
entered through external agents working 
with the school or educational vendors. 
External assistors worked with schools 



through highly formalized relationships with 
comprehensive school reform agents by one 
school in Florida or with educational 
evaluation teams by II/USP schools in 
California. Schools also worked with 
external agents informally. Schools in 
Michigan described receiving information 
from less formalized relationships with their 
regional education centers, and one school 
in California learned about a strategy when a 
vendor approached the school directly. 

When we typically think about 
individuals searching for solutions, we think 
about individuals engaging in research 
efforts, attending trade conferences, or 
talking with people who are doing different 
things. However, decision makers bring 
information into decision situations from 
sources both close and far. In this discussion 
we have differentiated between information 
within the school and outside the school 
because information from outside the 
organization has the potential to bring 
variation to the ideas discussed inside the 
school. We have explained how information 
gathered from within the organization might 
possibly offer variation when these ideas 
remain latent. While ideas from outside the 
organization offer great potential for 
introducing variation, it should be noted that 
those seeking information are often drawn to 
familiar and comfortable ideas (Rosenkopf 
& Almeida, 2003). For example, principals 
may be drawn to literature or conferences on 
after-school programs to remediate students 
instead of new remedial curriculum because 
they are already familiar with or have 
already implemented some form of tutoring 
program in the school. Therefore, external 
information does not always imply novel 
information. 

This discussion shows that, despite the 
continued reliance on local information for 
decisions being made in schools, the 
bounded rationality model is not purely 
playing out in the high schools we visited. 
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The constraints in accessing information are 
being overcome by active districts and, at 
times, the initiative of people inside schools. 
Since the model for search and selection 
suggests that constrained access to reliable 
information is a primary reason 
organizations focus on locally known 
strategies, the fact that many of the schools 
in our sample acknowledged a need to meet 
the state’s goals and showed channels of 
information coming into the schools 
suggests that they might also be willing at 
times to select strategies that deviate from 
the model’s second prediction and challenge 
the technological core of their school. We 
might expect that the avenues to new ideas 
would introduce strategies that offered a 
departure from the technical core of the 
school and that these ideas would 
occasionally be incorporated into their 
change effort. 

The Selection of Strategies: 
A Range of Possibilities 

The extent to which strategies deviate 
from the school’s traditional practice shows 
the potential the reform has to significantly 
change the educational experience of 
students in the school. While change is not 
necessarily good, it is probably fair to 
suggest that improvement in low-performing 
schools will be limited unless these schools 
change the way they work with students. In 
this chapter we cannot evaluate the impact 
of changes on student achievement, but we 
can comment on the extent to which we saw 
schools making changes to improve student 
performance. In the discussion above we 
suggested that the new information may 
introduce strategies that would change the 
work of the school. It is important to 
remember, however, that the model of 
bounded rationality does not account for the 
fact that in many schools new information 
that was used to make decisions was highly 



unsystematic, hurried, unfocused, and often 
pursued by individuals or small groups in 
the school. These issues, no doubt, lessened 
the impact information might have in the 
organization by limiting the scope of 
individuals processing the new ideas, 
hindering a clear direction for the 
information, and relying heavily on the 
fortunate coincidence of ideas and decision 
instead of more deliberate efforts to identify 
a need and seek a solution. These limitations 
notwithstanding, we found instances in 
which schools stepped away from the 
peripheral changes that dominated the 
reform efforts we saw in our sample. These 
cases, though not common, are significant in 
that they show the reform possibilities that 
can be realized in supported high schools 
with a motivated staff. In this section, we 
focus on the extent to which the strategies 
described by our respondents hold some 
potential to change the core technology of 
the school. Because this chapter focuses on 
instructional change, we define the core 
technology of the school as the curriculum 
(what is taught), instructional practice (how 
curriculum is taught), and organizational 
structure. In this section we intend to 
provide a sense of the range seen across 
schools and within schools. To do so, we 
focus on four strategy types that together 
reflect the vast majority of strategies 
described by respondents in our study: (a) 
remedial strategies, (b) curriculum 
strategies, (c) instructional practice, and (d) 
organizational strategies. Our discussion 
does not provide a complete breakdown of 
the strategies used in the schools we visited. 
A more thorough analysis of the specific 
approaches used in our sample schools can 
be found in Harris, Prosky, Bach, Heilig, 
and Hussar (in this publication). The most 
important point to take away from this 
discussion is that, despite theory that 
suggests that targets drive decision makers 
to select strategies that aim just to meet the 
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target, schools across all of our 
accountability contexts showed a range of 
responses that go from the “quick fix” or 
minimal-impact approach, which we call 
peripheral, to strategies that in modest ways 
change the educational experience of some 
students, which we call moderate changes, 
to more fundamental change in the 
educational program brought to students, 
which we consider significant changes to the 
core. 

Remedial Strategies 

The many add-on programs used in 
schools to remediate students operate on the 
periphery of the school’s core technology, 
anchor one end of the continuum of impact 
on the schools’ core technology, and 
represent more than half of all the remedial 
approaches pursued by the schools in our 
sample. For the most part, these programs 
did not interrupt the curriculum used by 
teachers or affect the instructional practice 
of teachers. These programs include before - 
or after-school tutoring programs or 
teachers’ efforts to meet students outside of 
the regular school day. While these 
programs generally offered students the 
opportunity to receive additional instruction 
from teachers (few schools used outside 
tutors or peer volunteers), schools usually 
did not or could not require students to 
participate in the programs. 

Schools also used remedial strategies 
that involved a new curriculum for the 
schools’ lowest performing students, and 
these efforts represent approximately 40% 
of the remedial strategies in our sample. In 
these cases, schools created remedial classes 
for their weakest students. Schools often 
created these courses for their 9 th grade 
students, but in some cases these classes 
included students through the 12 th grade. 
Schools typically assigned students to these 
programs on the basis of their test 



performance, but some schools used 
traditional student tracking mechanisms 
such as teacher evaluation. Schools intended 
most of the remedial courses recently 
introduced to provide an additional class for 
students to take before beginning the high 
school curriculum. While their impact on the 
school’s core technology overall was very 
limited, in these courses students 
experienced a specialized curriculum. In 
contrast with the add-on classes, one school 
in our sample rewrote the regular curriculum 
for its lowest level 10 th -grade English class 
to include a new chronology of information 
and new information. While still targeting 
only a set of students, this effort shows an 
even deeper impact on the core technology 
for these students. Only three schools in our 
sample went so far as to completely rethink 
the educational program they offered to their 
lower level students. 

Curriculum Strategies 

Peripheral strategies predominated the 
activities described for us, but unlike the 
remedial approach where we saw few 
instances of schools moving beyond 
moderately deep strategies, the focus on 
standards and standards alignment in the 
policy seems to have prompted several 
efforts to create deeper change in the 
curriculum. Well over half of the curriculum 
strategies were very peripheral or only 
moderate changes, which included the many 
ways teachers added test prep activities to 
their lessons individually or as part of 
department and schoolwide efforts, 
programs that added new advanced courses 
to the school’s curriculum, and efforts to 
introduce or expand advanced placement 
programs. However, almost a third of the 
strategies related to curriculum provided 
deeper curriculum change that involved 
efforts to rewrite curriculum and align the 
school’s curriculum to the state’s standards. 
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In some cases, the curriculum revision, 
initiated at both the school and district 
levels, resulted in significant changes to the 
timing and coverage of courses. 19 Finally, 
the most substantial effort to alter the core 
technology of a school was one school’s 
decision to adopt an entirely new math 
curriculum for grades 9-12 based on 
principles of integrated math, previously not 
a the central approach used by the school’s 
math department. This change came with 
not only a new philosophy of teaching math 
but also new materials, intensive and long- 
term professional development for teachers, 
and a battery of benchmark assessments. 
Students in this school after the 
implementation of the new curriculum 
experienced courses that were radically 
different from their predecessors. 

Instructional Practice 

The range of efforts to change 
instructional practice had individual self- 
initiated professional development by 
teachers, which accounted for two thirds of 
the instructional strategies in our sample, at 
one end of the spectrum, with schoolwide 
efforts to educate the entire staff in a 
specific instructional style on the other end. 
In between these examples of very 
peripheral and very substantial change 
efforts, popular efforts included attempts to 
provide professional development on 
teaching to standards as well as 
incorporating reading strategies and writing 



19 The efforts to rewrite curriculum show one way in 
which accountability may well have mitigated some 
of the information constraints faced by schools. 
Many respondents indicated that the standards to 
which they aligned their curriculum helped reduce 
the uncertainty and ambiguity surrounding what they 
should teach. In doing so, standards to some extent 
reduced the uncertainty associated with teaching and 
the educational process, which is one of the factors 
leading to a satisficing response, a response in which 
the decision making only aims to satisfy the targets. 



activities schoolwide. Both of these efforts, 
if enacted, challenge traditional norms of 
teacher autonomy and norms of 
differentiation in content, goals, and 
practices for different subjects. In addition, 
schools showed concerted efforts to 
introduce teachers to technology they can 
use in the classroom as well new approaches 
to engaging students. To varying degrees we 
found that these efforts did impact teachers’ 
practice. For example, teachers described 
changes they made in presenting material to 
students based on training from a district- 
sponsored consultant, teachers discussed 
instruction with literacy coaches, and 
teachers worked in clusters to jointly plan 
curriculum and lessons. 

Organizational Changes 

Organizational changes that affected the 
instructional program represent another class 
of strategies adopted across the schools we 
visited, though they were by far the least 
discussed by our respondents. Most of the 
attempted strategies included schedule 
changes or minor changes to the school day. 
The most peripheral changes to organization 
we found included efforts to build in 15 
minutes for silent reading or test prep. 
Changes to the school day, typically to the 
block schedule, represented change that had 
the potential to significantly change the way 
teachers present the curriculum to students. 
That potential is more likely to be realized if 
teachers received training for such change 
and were willing to make the change. As it 
turned out, the impact of this policy on the 
core tended to be limited. The institution of 
a ninth-grade academy represented another 
popular organizational change made in the 
schools we visited. The ninth-grade 
academy shows some effort to change the 
educational experience for students, but this 
effort was limited in the total number of 
students impacted and had relatively modest 
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implications for the instructional program. 
We did, however, visit a school that used a 
new organizational structure to support an 
instructional approach and thus offered an 
example of an organizational change that 
challenged the core technology of the 
school. This school, which had previously 
operated as a technical high school and had 
recently become a comprehensive high 
school with traditional academic disciplines, 
adopted a schoolwide model that 
complemented the vocational tracks with a 
restructuring of the academic program into 
interdepartmental teams. In addition, the 
school design introduced a curricular 
package intended to operate with the 
interdepartmental and the vocational 
orientation of the school. Although the 
design is still being implemented and will 
likely face some challenges as it rolls out, in 
many ways this school looked and acted 
very differently than the traditional high 
school. 

Looking across the schools we see a 
range of activities that show the need for 
immediate gains, possibly as a response to 
the press of accountability, as well as deeper 
efforts to improve the instructional program 
for their students. It is interesting to note 
that we saw this variation within schools as 
well, indicating that schools realize that they 
must show gains immediately to avoid the 
consequences of public reporting and 
possible sanctions but also realize the need 
for long-term improvement. Even sites that 
engaged in the most substantial change 
efforts described activities expected to 
achieve immediate gains by targeting low- 
scoring students with additional assistance 
and providing all students with test 
preparation activities while they 
simultaneously pursued curriculum 
alignment, writing across the curriculum, 
new instructional approaches or new 
organizational structures. A school in 
Florida provided an example of this blended 



approach to meet immediate testing needs 
by implementing programs to tutor low- 
performing students after school and new 
courses for ninth-grade students who scored 
below grade level in reading and math while 
simultaneously pursuing longer term 
improvement with a schoolwide effort to 
vertically align curriculum to ensure less 
overlap in course content and establish a 
track through which more students can reach 
honors-level courses. Although few schools 
ventured to the far right of the continuum 
shown in Figure 2, most of the schools in 
our sample showed blending of very 
peripheral strategies with strategies that 
made moderate changes to the core 
technology. 
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Figure 2. A Continuum of Change 
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Conclusions 

The goal of state accountability policy 
is to capture schools’ attention, direct 
schools’ focus, and motivate action. To a 
large extent our visits to 48 schools shows 
that the states have, in fact, succeeded in 
these goals and in so doing have an 
important presence in the decision making 
of schools. Though we saw exceptions in 
each state, teachers and administrators have 
incorporated the state’s goals into their own 
articulation of goals or sense of 
accountability. The problems and challenges 
identified by our respondents clearly reflect 
their concern over the state’s performance 
goals, and a high share of the problems 
highlighted by the states accountability 
reporting and sanctions were addressed in 
some way by the schools. While we do not 
have the longitudinal view that would allow 
us to say whether schools are engaged in 
more improvement than prior to the 



implementation of accountability, teachers 
and administrators in many of the schools 
we visited were engaged in efforts to 
improve student achievement. 

The success of these policies, however, 
also relies on the ability of schools to seek 
and select appropriate strategies for their 
schools. Accountability policies, by design, 
leave most of the decision making regarding 
reform initiatives to agents in the local 
schools and districts, who presumably have 
unique knowledge of their schools’ needs 
and strengths. States, by consequence, had 
little direct influence over who was making 
decisions in schools, over how schools 
organized to select strategies for 
improvement, or over the information used 
to make decisions. In the two states that did 
play a direct role, this support was reserved 
for a small number of the state’s lowest 
performing schools. As it turns out, in most 
of our schools the press generated by the 
state accountability systems seemed to do 
little to prompt schools toward more 
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coordinated or systematic decision-making 
efforts. Few schools engaged in deep needs 
analysis, few schools described thorough 
efforts to analyze current or potential 
programs, and schools did not consistently 
seek information on strategies or use 
research-based evidence in support of their 
decisions, a continuing hope in the 
Department of Education. In addition, 
schools only rarely made use of already 
existing structures to coordinate the search 
and selection of strategies. Only a few 
schools discussed the use of schoolwide 
planning teams for decision making. 
Departments, despite their importance in 
organizing teachers, provided the forum for 
significant decision-making efforts in only a 
few occasions. By and large, the 
departmental structure offered individual 
teachers a professional group within which 
they engaged in social and professional 
conversation, but the departmental structure 
did not represent an organizational form in 
which information was collected, discussed, 
and then decided upon. Schools often 
overcame the burden of seeking strategies 
by relying heavily on information already 
known in the school from prior professional 
development or professional experience and 
often selected strategies that did not 
radically change the way they had always 
worked. 

Given this final assumption, it was not 
surprising to learn that cases showing a high 
level of sophistication and coordination of 
the decision process also seemed to be 
places that had a history of such efforts, 
described as a “legacy” in one school we 
visited. We found that districts succeeded in 
introducing new ideas to schools and 
helping to facilitate more coordinated and 
deeper reforms in schools. While the 
instances in which the districts’ support led 
to comprehensive change that challenged the 
technical core of schools were in the 
minority of cases in our sample, the impact 



of districts in these instances was very 
promising, suggesting that the district may 
be the most effective support provider to 
schools that states may want to facilitate. 

We suggest that states think seriously about 
how to mobilize districts and facilitate 
district efforts to assist their schools. 

It is important, however, to remember 
that not all school districts have the 
resources to be active support providers. 
Many of the nation’s smallest districts have 
little more than a superintendent and 
administrative assistant occupying the 
district office. In these cases the most logical 
support structure to assist the schools are the 
regional educational centers which existed 
in some form in each of the six states we 
visited. These centers with pooled resources 
can serve as clearinghouses for information 
on new approaches, can coordinate 
evaluation efforts, and can house coaches 
and reform facilitators that serve several 
regional schools. Unfortunately, not all 
states use these organizations in the effort of 
schools reform. In a few states these 
organizations had unclear mandates 
regarding their role in supporting school 
reform, and in others the funding for these 
organizations had been so drastically cut that 
centers that districts did not financially 
support had been closed. Therefore, we 
suggest that states consider these regional 
centers as potential support structures for 
school reform in regions with small districts. 
We suggest that the state reconsider the 
mission of these centers to focus their 
resources around school and instructional 
reform and provide resources to these 
centers to effectively provide this support. 

In this chapter we described the ways in 
which state accountability policies shaped or 
did not shape the schools decision making 
with regard to improving student 
performance. We learned that that 
accountability policy has to a large extent 
shaped the focus of decision making but 
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“leaving the specifics to the locals” is not 
ideal when local conditions such as the 
quantity and quality of information about 
new strategies and the degree of 
communication and coordination in schools 
do not lead to effective decision making. 
This work certainly identifies weaknesses at 
the local level that hinder efforts to adopt 
and implement improvement strategies. 
However, this work also shows that high 
schools will make changes to their academic 
program and practice when the information 
about strategies is brought to them and they 
are supported through the change. 
Accompanying accountability policies with 
policies to improve information availability 
as well as mobilizing support structures such 
as districts or regional centers offers the 
chance to improve decisions and 
implementation in schools while remaining 
faithful to the spirit of outcomes 
accountability. 
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Chapter 4 

Overview of Actions Taken by High Schools to 

Improve Instruction 

Donna M. Harris, Melissa Prosky, Amy Bach, Julian Vasquez Heilig, and Karen Hussar 



Introduction 

Purpose 

Over the course of our fieldwork in 48 
high schools, researchers learned about a 
variety of different strategies utilized with 
the goal of improving instruction. The 
objective of this piece is to provide a 
descriptive overview highlighting actions 
that high schools were undertaking, framed 
by the issue of accountability. As stated in 
chapters 1 and 2, state-level (and in some 
cases district-level) accountability policies 
played a major role in the course of actions 
taken by high schools in their attempts to 
improve achievement. 20 

This chapter examines strategies for 
improvement around three key components 
of accountability systems: (a) standards and 
content requirements, (b) minimum 
requirements, and (c) data and assessment. 
To this end, we have directed the following 
discussion around three broad categories of 
strategies: (a) curriculum and instructional 
strategies to meet standards and content 
requirements, (b) remedial efforts to meet 
minimum requirements, and (c) efforts to 
make use of data and assessments. In each 
of these sections, we have examined the 
strategies used, and where available, the 
major supporters and avenues of support. 21 



20 The field visits took place during the 2002-2003 
school year; therefore, schools were only beginning 
to feel the pressures of NCLB. 

21 The discussion focuses on the initiatives that were 
most often cited by school staff in each of the 



We also recognize that high schools 
take on other responsibilities besides 
fulfilling accountability requirements. To 
this end, we have also included efforts 
around college and career preparation, 
college outreach, and parental involvement. 
These categories of strategies were 
commonly mentioned across the schools in 
our sample. 

Data Analysis 

The analysis for this paper relied on the 
case studies written by the research teams 
that visited each school. Researchers wrote 
detailed summaries based on interview 
transcripts and school documents for each of 
the schools and districts in the sample. The 
authors used the 48 high school case studies 
to create an Excel database of high school 
actions. This database contained qualitative 
information about the activities in which 
schools were engaged. We then assigned 
codes to each action to categorize the nature 
of activities that took place." The discussion 
here describes actions as reported by school- 
level staff but does not evaluate the quality 
of implementation or their effect on student 
achievement. 



categories. For a more complete list, see the tables at 
the end. 

22 The following list of codes was applied: curriculum 
and instruction, remediation, professional 
development, organizational/structural changes, 
support, assessment and data, and other. These 
groups were then integrated for the purpose of this 
paper. 
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Actions as They Relate 
to Accountability 

Curriculum and 
Instructional Strategies to 
Meet Standards and 
Content Requirements 

Schools were engaged in a variety of 
actions to help students meet the standards 
and content requirements of their work. 
Initially, we tried examining curriculum and 
instruction as separate categories. However, 
given the degree of overlap between the two, 
we decided to combine the areas. This 
section looks at initiatives in two major 
subcategories: classroom-based strategies 
and collaborative efforts among teachers. 

Classroom-Based Strategies 

This category is defined as actions 
employed by school staff members that 
could potentially change or impact 
instruction in the classroom. The most 
prominent strategy that we found across the 
48 high schools was the use of the block 
scheduling. At least a third of the schools in 
our sample had either a full block schedule 
in place, a modified block, had gotten rid of 
the approach, or were in the process of 
considering its implementation. Block 
schedules were often used as a way to get 
students to spend more time on tested 
subject areas, and used as a component of 
various reform measures. In some cases, we 
learned that professional development had 
been provided to teachers in schools that 
were adapting this approach. 

Another popular category of approaches 
used by high schools revolved around the 
use of test preparation. At least four 
buildings used “writing across the 



curriculum” as a way to improve their 
students’ composition skills in different 
subject areas for standardized tests. An 
additional intent of this initiative was to help 
broaden the responsibility among school 
faculty for improved student writing, rather 
than simply letting it rest with the English 
department. Some high schools made use of 
online sources for test preparation in math 
and English skills. Students at one 
Pennsylvania high school reported using My 
Access, an online writing tutorial that 
provided students with immediate feedback 
on their work, while those at a New York 
school used practice questions from a 
program called School Island, which cut 
across subject areas. One California high 
school even instituted a zero period test prep 
class, which was required of all freshmen. 
Finally, there were various motivational 
strategies used by schools to help improve 
test performance. Among these, an urban 
California high school held schoolwide 
assemblies to stress the importance of 
testing, while the principal of Pinewood, a 
similarly situated school initiated a 
campaign to improve the building’s 
Academic Performance Index (API) score. 
As part of this effort, students designed and 
posted motivational posters around the 
school to encourage their classmates to do 
well on the upcoming tests. 

Across the 48 schools, we saw a few 
examples of whole school reforms. These 
included America’s Choice, Comer School 
Development, and High Schools That Work. 
The introduction of these programs was 
made possible through the efforts of the 
buildings’ respective school districts. The 
approach that we saw most fully 
implemented was the America’ s Choice 
model at Tech High School in Florida. 
Curriculum and instructional components of 
the program included: Ramp Up, a reading 
program geared toward 9 th - and 10 th - grade 
students who were 1 to 2 years behind in 
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reading, the Foundations of Advanced 
Literacy ELA curriculum, and Foundations 
of Advanced Mathematics. 

Organizationally, America’s Choice 
incorporated block scheduling as well as an 
interdisciplinary team to break up the status 
quo departmental structure. Although High 
Schools That Work was designed to be a 
whole school reform, for one urban 
Michigan school, this was only one of a 
slew of initiatives taking place in the 
building. In practice, this approach existed 
as a single interdisciplinary team 
encompassing a handful of teachers, and 
was introduced to help raise student test 
scores. Finally, one North Carolina high 
school was in the beginning stages of 
implementing the Comer School 
Development Model, which represented a 
switch from High Schools That Work. At 
the time of our data collection, we learned 
that the school had secured funds for 
teachers to receive professional development 
around this program. 

Teachers and administrators mentioned 
various opportunities for professional 
development that were connected to 
instructional improvement. Some discussed 
attending conferences that were offered by 
the International Reading Association 
(IRA), the National Council of Teachers of 
Mathematics (NCTM), the National Council 
of Teachers of English (NCTE), and the 
College Board. There were also district- 
sponsored workshops covering topics 
including Kagan Cooperative Learning and 
manipulatives in math. One of the most 
deeply implemented district-supported 
curricular programs was Core Plus at a 
suburban high school in Pennsylvania. This 
program, intended to help increase student 
achievement in math, involved professional 
development that was both intensive and 
embedded. 

Researchers asked teachers and 
administrators about mentoring and 



classroom observations as part of the visit 
protocol. Respondents across all six states 
reported some type of mentoring for new 
teachers; however, these did not appear to be 
particularly consequential. For example, 
some schools would assign a veteran teacher 
to mentor a new colleague (sometimes in a 
different department), and this relationship 
would involve administrative tasks, rather 
than meaningful conversations about 
instruction. There was also little in the way 
of classroom observations among colleagues 
taking place, aside from formal evaluations 
by administrators. 

Collaborative Efforts to Improve 
Curriculum and Instruction 

In addition to the previously described 
strategies, we found a variety of 
collaborative efforts to improve curriculum 
and instruction in high school. These 
processes, normally undertaken by groups of 
teachers, were used to coordinate goals and 
objectives across departments and schools. 

School improvement planning was the 
most prominent collaborative strategy 
mentioned for improving curriculum and 
instruction. These planning meetings 
provided a forum for staff members to 
formulate goals, such as raising test scores, 
improving school climate, and collecting 
data. In Florida, this process was a state 
requirement for all schools. In other states, 
this was supported by districts and often 
used in conjunction with other activities, 
such as the North Central Association 
Accreditation Process in Michigan. 

Although a commonly mentioned approach 
by school staff, it often seemed to be merely 
an exercise in paperwork because of lack of 
collective follow-up. 

At least seven of the schools in the 
study reported that some type of curriculum 
alignment took place. This was a process by 
which teachers incorporated the state and, 
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where applicable, district standards into the 
curriculum. The degree to which this work 
took place varied among our sites. Teachers 
based at high schools in Michigan had 
access to the online program MI CLiMB 
(Clarifying Language in Michigan 
Benchmarks). This state-designed program 
was designed to help teachers align 
curricular topics to the state benchmarks and 
link their data to the state curriculum. The 
two high schools that we visited in Hampton 
City, an urban Michigan district, used 
pacing guides, which the district had 
instituted as part of its curriculum 
realignment. The purpose of these guides 
was to ensure that teachers were covering 
essential areas. An example of curriculum 
alignment occurring on the school level took 
place at California’s Pinewood High School. 
At this site, departments devoted much of 
their collaboration time to revising and 
aligning their curriculum to state and district 
standards. Similar to alignment, schools 
reported engaging in curriculum mapping, 
whereby teachers worked together in 
outlining what was going to be taught in 
each class. This tended to occur in math 
departments, among teachers who were 
teaching the same course. At Mission High 
School, located in an urban Florida district, 
clusters of math teachers of the same course 
collaborated on scope and sequence, 
creating common exams and selecting 
textbooks. Respondents at a California 
school reported that a district math 
consultant visited the math teachers 
approximately every other month to help 
map their curriculum. This was a young 
department which was viewed as having 
needed the outside assistance. 

Other reported collaborative efforts 
around curriculum and instruction, which 
helped support accountability systems, were 
mentioned by respondents in fewer 
buildings. One of these was actions to 
establish periodic curriculum cycles in math 



and English. One Pennsylvania school 
district, for example, established a 3-year 
revision cycle. As part of this initiative, 
students were given twice-a-semester 
milestone tests that corresponded to content 
coverage by quarter. At the time of the field 
visit, these were being given in the math 
department and were set to begin for 
English. Another effort mentioned by some 
teachers was the existence of district- wide 
curriculum councils. In a California district, 
for example, all of the buildings, including 
the high school, provided representation to 
their curriculum council. The council served 
as one of several bodies that provided 
recommendations to the school board for 
final approval. 

Remedial Efforts 

Remediation was a major strategy 
employed by high schools to bring their 
students up to the minimum state 
accountability requirements. The chief 
methods of remediation we found were 
through a modified curriculum in regular 
classes, separate remedial classes, and 
tutoring. 

Modified Curriculum in Regular 
Classes 

One of the major findings in this study 
was the focus on reading in high schools. To 
this end, we learned about several software 
packages that were used as part of the 
regular classroom setting to help remediate 
low-performing students. Two high schools 
in Pennsylvania, both located in the same 
urban district, targeted the Academy of 
Reading program for use with ninth-grade 
students who needed the extra assistance. 
The goals of this program included 
improving students’ technical reading and 
reading comprehension. A district literacy 
coach was responsible for implementing the 
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use of this software program throughout the 
district. Training and technical support for 
the program was provided by the company 
Point Click Learn. Another reading 
program, called Accelerated Reader, was 
used at Pinewood High School and a 
building in North Carolina. Through this 
program, students took a series of tests, 
using different texts. The results were then 
used to determine when the student was 
ready for the next level. 

To help support literacy efforts in the 
classroom, a number of schools brought in 
literacy coaches to work with teachers and 
students. Two urban California schools, both 
located in the same district, had a district- 
sponsored outside consultant help teachers 
with reading across content areas. Another 
California building reportedly formed a 
reading department to serve this same 
purpose. Florida’s Mission High School and 
Harbor High School, also located in the 
same district, received assistance from the 
district reading coordinator. This individual 
visited the buildings once a week to meet 
with teachers, to assist them with use of 
CRISS (CReating Independence Through 
Student-Owned Strategies) strategies and, as 
needed, to model lessons. 

We also learned about a few 
computerized remedial programs being used 
in the area of math. By far, the most 
commonly cited one was Cognitive Tutor, 
developed by Carnegie Learning. This 
program provided help to students who had 
difficulties in traditional math classes, 
including Algebra I and geometry. The 
schools utilizing this program were 
concentrated in New York, North Carolina, 
and Pennsylvania. Where this information 
was known, it was the district that had 
brought this program to the schools. Other 
remedial math programs mentioned by 
schools included Accelerated Math and I 
Can Learn. 



Separate Remedial Classes 

To help bring students up to the 
minimal standards, high schools made 
available a number of remedial options to 
bring their students up to speed. These 
included classes in math and reading, plus 
broader opportunities that targeted low- 
performing students. 

The major concern regarding high 
school math was the focus on getting 
students to pass Algebra I. All of the states 
in the study included algebra as part of their 
standardized assessments. The most 
common approach to dealing with this issue 
was by stretching out a semester’s worth of 
algebra over the course of a school year. 
Algebra IA and Algebra IB would be 
completed in 2 years, instead of the 
traditional single year. At least four of the 
visited high schools in North Carolina 
utilized this extended course of study to help 
students fulfill the state graduation 
requirement in algebra. In Michigan, the 
math department at one high school 
instituted a second-semester repeat algebra 
class. This appeared to be mandatory for 
students who had failed the first semester. A 
final example of changing Algebra I was 
seen at Oceanside High School in Florida. In 
this case, the school offered a double-period 
Algebra IA/IB “combo” class, which 
targeted at-risk ninth- grade students. 

We learned about separate remedial 
reading classes in high schools, most of 
which were concentrated in North Carolina. 
In at least four schools (three of which do 
not overlap with the schools mentioned in 
the previous paragraph), targeted students 
took reading as an elective course in 
addition to their regular English class. 
Students were identified for this class on the 
basis of failing to meet proficiency on the 
eighth-grade state reading test. 

Florida’s Oceanside High School was a 
school that stood out due to the sheer 
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number of reading classes offered in the 
building. A total of 60 classes, ranging from 
the third-grade reading level through 1 year 
below grade level, were made available for 
students. Because of the state accountability 
grade received from Florida, this school 
received significant assistance from the 
district office. Apparently, the district 
applied for a comprehensive school reform 
(CSR) grant, and had the funds directed to 
this particular building. Through a 
collaborative approach with school 
leadership, it was determined that remedial 
reading classes should be instituted as a way 
of alleviating this recognized challenge. 

Other schools made use of additional 
approaches to assist at-risk students. An 
urban high school, for example, had a 
program designed for overage ninth graders 
to receive instructional support on the 
campus of North Carolina A&T. Students in 
this programs took classes including Success 
101, English, math, history, and science. A 
Florida building had a program called Leap 
Forward to help students who had fallen 
behind make progress toward graduation at 
an accelerated rate. The school arranged a 
course schedule that allowed students to 
make up significant deficiencies in credit 
requirements. This program allowed 
students to develop alternative schedules to 
attend school part-time or earn credits in off- 
campus courses. 

Tutoring 

Respondents from just about every 
school in the study mentioned tutoring as a 
way of providing extra help for students. 
This was usually offered to individual 
students on a voluntary basis after school. In 
a few cases, students received tutoring from 
college students at nearby universities, and 
in other cases, tutoring was made available 
on Saturdays. 



Making Use of Data and 
Assessments 

A final category of actions examined 
relating to accountability was around the use 
of data and assessments. We saw few 
examples of high schools collecting and 
otherwise employing their data. When asked 
about this, most teachers and administrators 
would mention discussing the results of the 
state assessment at a school or department 
meeting, but little follow-up beyond that. 
This section discusses the handful of 
instances where there seemed to be 
additional efforts to collect and utilize data 
to inform instruction. Actions in this 
category include the use of diagnostic tests, 
progress assessments, a comprehensive 
database, an outside vendor program, and 
on-site personnel. 

After years of frustration of having 
students misplaced in classes (e.g. students 
receiving credit for Algebra I in middle 
school without having learned the material), 
the math department at Mission High School 
decided to create a diagnostic test. This test 
targeted all incoming students to ensure that 
they were appropriately placed. Other 
schools used diagnostic tests in conjunction 
with progress assessments to track student 
achievement. A California high school, for 
example, used Northwest Regional 
Educational Laboratory (NWREL) 
assessments in math, reading, and writing. 
These were administered to students three 
times a year (beginning, mid-year, and year- 
end) to help place students and measure 
their growth. Another school, located in the 
same state, administered the Stanford 
Diagnostic Reading Test. Like the NWREL 
assessments, this one was also given three 
times a year to gauge student progress. This 
test targeted incoming ninth-grade students 
to help place them into appropriate classes. 

Faculty at two Michigan high schools, 
both located in the Hampton City School 
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District, mentioned the district-created 
quarterly assessments. These assessments 
corresponded to the district’s pacing guides 
and were instituted as part of the effort to 
realign the curriculum across all grade 
levels. Although the use of the quarterly 
assessments was mandated by the district, it 
was up to school-level administrators and 
department heads to monitor their use. Some 
teachers did resist using this tool, because 
they were not sure how these data were 
going to be used. 

An urban Florida district created a 
comprehensive database containing 
demographic and student assessment records 
for all students. Although the goal was to 
have teachers use these data to make 
informed decisions about their instruction, 
teachers at the district’s Oceanside High 
School reported making very minimal use of 
this information. 

There was one suburban California site 
in the study that discussed using an external 
vendor for using data. At the urging of the 
principal and central office, the school used 
the web-based assessment platform available 
from Edusoft. This program was used for 
tracking student performance on the 
California state standards for three kinds of 
tests: state exams, district benchmarks, and 
in-class teacher tests. This was used to help 
inform instruction and chart students’ 
academic progress. 

Some schools had personnel based on- 
site for the purpose of supporting data use. 
For example, at a rural California school, 
one of the educational planning specialists 
was charged with analyzing data. Florida’s 
Harbor High School had an in-house “test 
chairperson” who managed and interpreted 
district and school data. It is noteworthy that 
for staff working with data, this was only 
one of many responsibilities on their agenda. 



Actions Beyond 
Accountability 

Although we found that most of the 
actions used by high schools were in 
response to accountability, it should be 
stated that schools strived to fulfill 
additional goals. These included the 
provision of challenging programs to 
prepare students for higher education, 
preparing students for employment after 
high school graduation, and increasing 
parental involvement. 

Magnet Programs 

A few sites we visited had magnet 
programs located within the high school 
building. Part of the purpose in devising 
these programs was to attract high-achieving 
students from around the school district, 
with an application process required for 
entrance. At Mission High School, students 
could choose from among academies 
focusing on business and technology, the 
arts, and liberal arts and sciences. One 
Michigan high school housed a well- 
regarded performing arts academy. Finally, 
Oceanside High had an International 
Baccalaureate (IB) program, which had been 
in place for several years. We were told that 
when the program was first introduced, 
Oceanside was the only school in the district 
to have it. Over time, however, other schools 
brought it in, which led to some competition 
across the district for high-performing 
students. Some staff members admitted that 
these students were helping the school to 
avoid the label for Florida’s lowest grade in 
the state accountability system. 
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Advanced Classes 

Most schools across the six states 
provided high-achieving students with 
opportunities to take advanced-level classes. 
For example, students could dual enroll in 
courses at local institutions of higher 
education. There were also Advanced 
Placement (AP) courses at many sites. Some 
teachers reported attending AP conferences, 
which provided information about teaching 
at this level. Our data collection did not 
provide detailed information about which 
subject areas these classes covered or how 
many students took advantage of them. 

College Outreach 

We learned about different measures 
that schools were taking to help direct 
students, particularly minorities, towards 
college. An urban New York school ran an 
initiative called Gateway to Higher 
Education, which was intended to help 
prepare high- achieving minority students 
for college and careers in the sciences. 
Mission High School held a forum so that 
students to could learn more about college 
from the alumni of the school. Harbor High 
School, located in the same Florida district, 
partnered with a several area organizations 
as part of the ENLACE (Engaging Latino 
Communities for Education) program, a 
partnership was to increase the number of 
Latino students graduating from high school 
and college. Areas of focus included 
tutoring, test preparation, and mentoring. 

Career Preparation 

Although they were not a direct focus of 
our field research, we did learn about some 
programs that schools had in place for 
preparing students for future employment. 
High schools in both Michigan and Florida 



allowed students to job shadow 
professionals in their area of interest. Other 
schools had variety of vocational education 
options available to students. At a New York 
school, for example, students could take 
classes in areas including architecture, 
communications, and culinary arts. A North 
Carolina school had a technology trade 
program in which students could dual enroll 
at the local community college while taking 
technology classes at the high school. 

Parental Outreach 

A final area that schools discussed 
beyond accountability was parental 
outreach. High school tends to be the grade 
level where parents are the least involved, 
and schools were trying out different ways 
to combat this. Two rural North Carolina 
schools, both located in the same district, 
had an initiative called Face to Face to 
encourage communication between teachers 
and parents. Parents had the opportunity to 
meet with their children’s teachers four 
times during the academic year. Twice a 
year, students led these conferences and 
shared their portfolios with their parents. A 
California high school initiated a parent 
institute, which we were told included 300 
participating parents during the year prior to 
the field visit. This 10-week program 
(parents came once a week for 10 weeks in 
the morning or the evenings) provided 
parents with information about different 
aspects of the high school and the services 
available for their children. This program 
was especially relevant for parents from 
different cultural backgrounds, who may not 
have been familiar with the United States 
school system. Another high school in the 
state had a parent technology training 
program. This initiative, which reportedly 
involved over 50 parents, encouraged the 
use of technology and English language 
acquisition in the home. 
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Table 1 . Curriculum and Instructional Strategies to Meet Standards 



and Content F 


Requirements 


Strategy 


Description/Purpose 


Web Site 


America’s Choice 


Prepare students for state and 
local assessments, and for 
college. 


http://www.ncee.org/acsd/program/high.jsp 


Block Scheduling 


Spend more time on tested 
subject areas for students to 
learn concepts, minimize 
disruptions. 




Comer School 
Development 


Connect child development 
with academic success. 


http://info.med.yale.edu/comer/ 


Core Plus Math 
Curriculum 


Curriculum to help students 
master math standards. 


http ://w w w . wmich. edu/epmp/ 
http://www.glencoe.com/sec/math/cpmp 


CRISS Strategies 
(CReating 
Independence 
Through Student- 
Owned Strategies) 


Help teachers develop 
strategies for improving 
student learning. 


http://www.projectcriss.com 


Curriculum 

Alignment, 

Curriculum 

Mapping 


Incorporate standards into the 
curriculum. Includes class 
clusters and use of pacing 
guides. 




Curriculum Cycle 


Periodically update the 
curriculum. 




Department Chair 
Off-Periods 


Develop curriculum and share 
with the department. 




District-Wide 

Curriculum 

Council 


Give teachers a voice in 
shaping the curriculum, 
working with colleagues in 
other schools. 




Freshman 

Academy 


Give ninth graders a separate 
space in the school, more 
personalized attention. 




Fligh Schools That 
Work 


Increase expectations, prepare 
students for college and work. 


http ://w w w . sreb . org/progr ams/hstw/hstwind 
ex.asp 


Literature Circles 


Prepare students for state tests 
by having them read in small 
groups. 




Math Journal 


Help students understand 
concepts, rather than engage 
in rote memorization. 




MI CLiMB 
(Clarifying 
Language in 
Michigan 
Benchmarks) 


Help Michigan teachers align 
curricular topics to state 
benchmarks. 


http://www.miclimb.net/ 
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Table 1 (continued). Curriculum and Instructional Strategies to Meet 
Standards and Content Requirements 



Strategy 


Description/Purpose 


Web Site 


My Access 


Writing tutorial that provides 
immediate feedback to 
students. 


http://www.vantagelearning.com/ 


National Board of 
Professional 
Teaching 
Standards 


Strengthen teaching 
standards. 


http://www.nbpts.org/ 


North Central 
Association 
Accreditation 
Process 


Help schools meet higher 
standards. 


http://www.ncacasi.org 


School Island 


Provide practice test 
preparation in math, English, 
science, and social studies. 


http://www.schoolisland.eom/review/login.a 

sp 


School 

Improvement 

Planning 


Give school the opportunity 
to formulate goals, including 
raising test scores and 
improving climate. 




Teaming 


Improve teacher 
collaboration. 




Thinking Maps 


Help students answer 
different kinds of questions, 
and track their thought 
process in reading and writing 
instruction. 


http://www.thinkingmaps.com/ 


UC-Irvine 
Collaborative 
Writing Project 


Help teachers improve their 
writing instruction. 


http ://www .gse. uci.edu/uciwp/ 


Validated 
Instructional 
Practice (VIP) 


Program with multiple 
components, including having 
teachers follow certain 
practices in every class and 
administration of mini-tests. 




Visual, Equation, 
Solution, Answer 
the Question 
(VESA) 


Rubric for helping students 
improve their math skills — 
along the same lines as 
writing across the curriculum. 




Writing Across the 
Curriculum 


Improve writing across 
subject areas. 
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Table 2. Remedial Efforts 



Strategy 


Description/Purpose 


Web Site 


Academy of 
Reading 


Software program to improve 
reading skills. 


http://www.autoskill.eom/products/reading/i 

ndex.php 


Academy 

Programs 


Provide extra attention to at-risk 
ninth-grade students. 




Accelerated 

Math 


Software with individualized 
lessons to improve math skills. 


http://www.renlearn.com/am/ 


Accelerated 

Reader 


Software to help teachers monitor 
reading progress. 


http://www.rcnlearn.com/ar/overview/dcfau 

lt.htm 


Carnegie 

Math/Cognitive 

Tutor 


Software with individualized 
lessons to improve math skills. 


http ://w w w . carnegielearning . com/start. cfm? 
startpage=products/ 


Compass 

Learning 

Software 


Software programs providing 
extra assistance across various 
subject areas. 


http://www.compasslearning.com/ 


Grade Recovery 
Course 


Prevent student dropout by 
allowing students who failed a 
marking period to improve their 
grade to a C by attending after 
school sessions. 




I CAN Learn 
(Interactive 
Computer 
Aided Natural 
Learning) 


Computer-based program to help 
students with algebra skills. 


http //www . ic anlearn. com/ 


Leap Forward 
Program 


Help students who have fallen 
behind to make progress toward 
graduation at an accelerated rate. 




Literacy 

Coaches 


Work with teachers and students 
to bring reading strategics into the 
classroom. 




Modified 
Algebra Classes 


Includes expanding one semester 
of algebra into two, double -period 
algebra, and the use of Integrated 
Math. 




NCE English 


Bring students up to standard in 
English. 




NovaNET 


Courseware to assist struggling 
students in meeting the standards. 


http://www.pearsondigital.com/novanet/ 


Read 180 


Software to help improve reading 
skills. 


http://teacher.scholastic.com/products/readl 

80/ 


Reading Classes 


Stand-alone classes to improve 
students’ reading skills. 




Tutoring 


Assist students with their work, 
normally on a voluntary basis. 





91 





Table 3. Data and Assessment Actions 



Strategy 


Description/Purpose 


Web Site 


Diagnostic Tests 


Help place students in 
appropriate classes. 
Examples included the 
Northwest Regional 
Educational Laboratory 
assessments, Stanford 
Diagnostic Reading Test. 


http://www.nwrel.org/assessment/ 


Database of Student 
Information 


District-designed system to 
give teachers access to 
student background data. 




Edusoft 


Web-based platform to help 
schools track assessment 
performance. 


http://www.edusoft.com/login.jsp 


Quarterly Assessments 


Track student progress over 
the course of the school 
year, make sure teachers 
are following curriculum. 




School-Based 

Personnel 


Coordinate, analyze, and 
manage data at the school. 
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Table 4. Actions Beyond Accountability 



Strategy 


Description/Purpose 


Web Site 


African 

American 

Student 

Outreach 


Invite African American leaders to 
encourage students to enroll in 
more demanding classes. 




AP Classes 


Rigorous course for college prep. 


http://apcentral.collegeboard.com/ 


College Forum 


Alumni visit school to discuss 
their college experiences with 
students. 




Dads and 
Donuts 


Increase male parental outreach. 




Dual 

Enrollment 


Provide more options for students. 




ENLACE 

(Engaging 

Latino 

Communities 
for Education) 
Program 


Increase graduation rates among 
Latino students. 


http://www.wkkf.org/Programming/Overv 

iew.aspx?CID=16 


Face to Face 


Increase parental involvement 
through conferences. 




Gateway to 
Fligher 
Education 
Program 


Prepare high-achieving minority 
students for science careers. 




Job Shadowing 


Allow students to see professionals 
in their field of interest. 




Magnet 

Programs 


Attract high-achieving students. 
Programs include academies and 
the IB program. 


http ://w w w .ibo . org/ibo/index. cfm 


Parent Connect 


Provide parents with greater access 
to their child’s information. 




Parent Institute 


Ten-week program for parents to 
inform them about the high school. 




Parent 

Technology 

Training 

Program 


Program to increase parents’ 
computer skills. 




STRIVE 

Program 


Partnership with outside 
organizations to provide mentoring 
for students. 




Test Nights 


Present parents with information 
about the state test and encourage 
them to provide home preparation. 




Vocational 

Education 

Opportunities 


Give students options to pursue 
coursework linked to career 
opportunities. 
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Chapter 5 

Stuck in the Middle With You: District Response 

to State Accountability 

Elliot H. Weinbaum 



Introduction 

State-mandated, performance-based 
accountability systems depend largely on 
theories of motivation that argue that 
schools will alter their practices in order to 
meet carefully defined outcomes that merit 
reward and recognition. At the very least, 
proponents argue, schools will aspire to 
demonstrate achievement in order to avoid 
increased state intervention, negative 
publicity, and a loss of professional 
autonomy. Such accountability systems 
became extremely popular over the last 
decade, and while they have undergone 
some significant changes in terms of the 
particular aspects of the systems, the overall 
theory and structure have remained 
unchanged. Although only a handful of 
states were using such performance-based 
systems in the mid-1990s, currently all 50 
states have adopted policies that follow the 
model just described (Goertz, Duffy, & 
Carlson-LeFloch, 2000). 

Analysis of performance data resulting 
from state assessment systems shows that 
while many elementary schools have seen 
significant strides in educational 
performance over the last decade, high 
schools continue to lag behind. A 
combination of increased focus on early 
education research and resources, smaller 
gaps to address at those early stages, and the 
more unified and uniform nature of 
elementary schools has allowed educators in 
the early grades to amass gains in 



achievement. Some argue that a portion of 
these gains can be attributed to state 
accountability systems that have set 
standards, focused attention, and created 
incentives for improved performance 
(Camoy & Loeb, 2004; Grissmer & 
Flanagan, 1998). High schools have not 
experienced the same positive effects. 
External indicators such as the Third 
International Mathematics and Science 
Study (TIMSS) and the National 
Assessment of Educational Progress 
(NAEP) regularly point to a system that 
loses ground as students progress in their 
educational careers. In part, this may be due 
to the relatively minimal attention that high 
schools have received until recently from 
state departments of education and state 
accountability systems. While there is 
undoubtedly some merit in postponing 
accountability for high schools until better- 
prepared elementary students advance to the 
high school level, the lack of attention to 
instruction and outcomes in high schools has 
stifled improvement. 

Regardless of the levels of attention that 
schools receive or the amount of 
accountability pressure that they feel, the 
strides that schools at all levels are able to 
make are heavily dependent on the resources 
for improvement that are available to them. 
School districts, also referred to as local 
education agencies (LEAs), are frequently 
cited as the most logical venue for providing 
assistance to significant numbers of schools. 
The federal No Child Left Behind Act 
(NCLB) seeks to increase the role that 
districts play in providing assistance and 
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monitoring performance. However, state 
accountability policies and the research on 
those policies have traditionally overlooked 
the role of school districts. Little research is 
available about the ways in which districts 
respond to accountability pressure or, until 
recently, the strategies that they might use 
for improvement. Much of the research that 
does examine the district role in school 
improvement or reform has focused on the 
elementary school level. High schools, with 
their distinct and somewhat autonomous 
departments, present districts with very 
different challenges. 

Because of the limited investigation that 
has been done, and the urgent need for high 
school improvement, I have chosen to focus 
on the state — di strict— school 
relationship. Through case study analyses, I 
have documented how districts devise 
different strategies, based in part on the 
accountability policies in their respective 
states, to help high schools to meet the 
challenges posed by state policy. This 
chapter argues that the district has a vital 
role to play in building capacity in all 
schools under accountability pressure. 
However, the extent to which districts fulfill 
that role is dependent upon a combination of 
variables both within the state policy and 
within the local district context. 

In this chapter, I briefly review previous 
research on districts’ function and their role 
in school improvement. I then describe my 
research to assess district role in a variety of 
contexts. I describe particular principles for 
assessing the quality of district initiatives. 
My research in 12 districts in two states 
found that districts could generally be 
divided into two “types,” which I have 
labeled A and B. Each of these types 
demonstrates certain behavioral traits that 
are more likely to lead to improved 
performance in the state assessment system. 
Finally, on the basis of this research, I 
provide policy recommendations to enhance 



the role of districts and improve their 
response to state policy pressures. 

Local district context can often be as 
much a factor in district activity and 
decision making as the state policy to which 
local actors are ostensibly responding. It has 
become common to state that “context 
matters” (Fuhrman, Clune, & Elmore, 1991; 
McLaughlin, 1991) to the way in which 
schools and districts implement policies and 
practices. The particular elements that 
comprise the “context” of LEAs are not 
always particularly clear. In conducting this 
research, I sought to specify elements of 
district context that are important in local 
implementation of state policy. Marsh 
(2000) has conducted a review of research 
on district-state relations. On the basis of 
this review, she identified six contextual 
factors of districts that may help to explain 
district response to state policy. The 
contextual characteristics that she identified 
are capacity, size, understanding, leadership, 
organization and governance, and political 
culture and reform history. Borrowing from 
Marsh, I use demographics, leadership, 
organization, and culture and history as 
contextual variables. Building on Marsh’s 
work and drawing on research by Elmore 
(2003), I define capacity (one of Marsh’s 
characteristics) to include the knowledge, 
skills, and resources that exist within the 
district. Additionally, on the basis of 
previous research about response to state 
accountability policy (Debray, Parson, & 
Avila, 2003), I add the history of test 
performance as an essential element of 
district context that will impact whether and 
how a district responds to state policy in 
general and accountability policies in 
particular. In the following section, I define 
each of the characteristics as they were used 
and analyzed in this research. 
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District Characteristics 

• Demographics (including district size). 
Community and student demographics 
have frequently been found to impact the 
ability and willingness of schools and 
school districts to engage in ambitious 
reform (Anyon, 1997; Lipman, 1998; 
Thernstrom, 1991). Similarly, research 
has found demographic issues to 
influence student performance (Fetler, 
1989; Howley & Bickel, 2000; Natriello, 
McDill, & Pallas, 1990). This becomes 
particularly important when discussing 
accountability systems, because the 
extent of state-district interaction that 
occurs within the framework of the state 
system is frequently based on school and 
district performance. Additionally, one 
of the demographic features that have a 
bearing on the district’s ability to 
respond to the pressure it faces is the 
size of a district, both the number and 
size of schools as well as the number of 
central office staff. While most agree 
that size impacts district function, 
researchers differ on whether bigger or 
smaller districts are better for schools. 

• Leadership. Leadership that is focused 
on well-defined, instruction-related 
issues over an extended period of time is 
most likely to succeed with 
implementation of policy and 
improvement in the district. Research 
describes the way in which school 
leaders who take the “opportunity” that 
accountability pressure can provide and 
use it to restructure existing leadership 
norms can demonstrate significant 
success in changing system behavior in 
instructionally effective ways (Lemons, 
Luschei, & Siskin, 2003). While 
educational research has recently taken a 
more favorable view of “distributed 
leadership” in which power, authority, 



and responsibility do not reside in a 
single individual but are encouraged at a 
number of levels of the organization 
(Spillane, Halverson, & Diamond, 

2001), this chapter does not regard 
distributed leadership as an empirical 
benefit but assesses the style of 
leadership and its match with district 
activities. 

• Organization. The organization of the 
district needs to support both the model 
of leadership and the goals being 
pursued. Frequently, a lack of 
communication between units of the 
district is responsible for incoherent 
improvement efforts. Occasionally 
because of historical causes, cultural 
habits, or resource constraints, within- 
district segmentation is a challenge for 
central offices engaged in system-wide 
improvement. 

• Culture and history. This characteristic 
attempts to capture the turmoil of local 
politics and historical divisions at the 
district level. Historical events that can 
impact district culture and approaches to 
reform include previous experiences 
with change, the alignment between the 
demands of state policy and the stance of 
the local district with regard to those 
demands, and the culture of the local 
community. 

• Knowledge, skills, and resources. As 
described previously, accountability 
response depends heavily on the 
marshalling and manipulation of 
knowledge, skills, and resources by 
agents to accomplish the tasks required 
of them. This means that an essential 
characteristic impacting district response 
is the degree to which knowledge, skills, 
and resources that can support that task 
exist in a district. In their discussion of 
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human, social, and physical capacity, 
Spillane and Thompson (1997) describe 
the ways in which the knowledge and 
skills of individuals within the system 
and the resources devoted to materials, 
personnel, and facilities play a key role 
in determining the ability of districts to 
respond to policy demands. 

• Performance. In an accountability 

system, school and district performance 
determines state action, which influences 
districts’ incentives to respond. As a 
result, it is extremely important to 
understand a district’s previous 
performance history in order to have an 
understanding of particular aspects of 
state policy with which the district may 
be engaged or have some experience. 

Districts or central offices are very 
much caught between the demands of state 
policy and the constraints and/or supports of 
local context. Districts with struggling high 
schools must respond in some way to the 
state policy in order to avoid sanction. 
However, districts are only willing or able to 
do so given the characteristics (as defined in 
the preceding) of the districts themselves. 
Assessing the simultaneous impacts of both 
policy and context is a challenge. The 
research community has relatively little 
understanding of the ways in which state- 
level, performance-based accountability 
systems and local school districts interact 
given various contexts. In no small part, this 
is due to the fact that many of the state 
accountability systems focus on schools and 
students rather than districts. As a result, 
much of the research has looked at the 
school level, and not investigated the ways 
in which such policies influence districts. In 
addition, the task of analyzing the broad 
contexts of districts, as opposed to a single 
school, is daunting. 



Research Questions 

With these relevant challenges in mind, 

I have developed several research questions 
to guide my study of these issues. The 
research questions I sought to answer are the 
following: Is there evidence that state 
accountability policies lead districts to 
engage in practices that are likely to result in 
instructional improvement at the high school 
level? If so, how? Do the type of state 
accountability policy and/or the particular 
characteristics of the districts significantly 
influence the district role in instructional 
improvement in high schools? These 
questions seek to assess whether districts are 
responding to state pressure in ways that are 
targeted at and likely to improve high school 
instruction and student performance. If such 
a response exists at the district level, I am 
interested in learning which levers at the 
state level and context variables at the local 
level combine to produce such a response. 
Additionally, I am studying whether the 
particular policy design choices that states 
make are likely to have a significant impact 
on the role of the central office. 

The literature that informed my thinking 
about these issues and the design of this 
study is composed of three parts: research 
on accountability, district role, and high 
school response. The general theory of 
action that supports the “new” educational 
accountability identifies student 
achievement as the primary goal of 
schooling and the focus of measurement and 
oversight. Achievement is measured by 
standardized assessments, and performance 
data is provided to students, parents, 
teachers, administrators, and members of the 
community. Stakes are attached to 
performance on these assessments, and the 
combination of information and stakes is 
used to motivate agents. The theory posits 
that agents in the system (most frequently 
teachers and students) will then work harder 
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(or smarter) in the areas the state has 
designated as important. Staff will either put 
previously acquired skills to better use or 
will seek to acquire the knowledge and skills 
that they feel will allow them to achieve the 
designated goals (Fuhrman, 2004). This 
means that the goals, standards, assessment 
mechanisms, data sources, stakes, and 
potential support all play a vital role in the 
ways in which this theory of action is 
enacted in the ongoing work of schools and 
school districts. 

Using this understanding of the current 
educational accountability models, I based 
my analysis of two state accountability 
systems on the work of Adams and Kirst 
(1999). The framework they offer describes 
performance-based accountability systems 
in terms of principal-agent theory. It asks 
questions about six elements of 
accountability policies: Who is identified as 
the agent? How is action authorized? How 
is agents’ productivity managed? How are 
accounts defined? How is compliance 
promoted? And how does the principal (the 
state in this case) ensure causal 
responsibility? This framework permitted 
me to examine the allocation of authority, 
standards at the high school level, 
assessments being used in the states, targets 
that schools and districts have to reach, 
incentives used to encourage actors to meet 
those targets, and the provision of resources 
by the state in order to help schools and 
districts reach their targets. Accountability 
policies that follow this design are rooted in 
the ideas of standards-based reform and 
systemic school reform as described by 
Smith and O’Day (1991). They are meant to 
align standards and assessments to focus 
students, teachers, schools, and districts on 
particular contents and competencies that, at 
least in this case, are of particular concern to 
state policymakers. 

The question of what role the district 
generally plays in moderating the influence 



of state policies at the school level is 
somewhat open for debate. Some 
researchers have claimed that districts are 
barriers to the type of school improvement 
that accountability advocates envision, 
arguing that districts’ incentive and 
organizational structures impede such 
sustained and meaningful reform (Chubb & 
Moe, 1990). Finn (1991) has argued that 
state-level accountability policies which 
possess the elements that Adams and Kirst 
(1999) identify obviate the need for school 
districts. That is, once the state sets 
standards and assessments (and perhaps 
provides resources for schools to get 
assistance), districts become an unnecessary 
governmental entity, not necessarily 
obstructionist, but certainly not possessing 
any significant utility. This proposition 
assumes that state policy is a “zero-sum” 
game in which the growing state role takes 
over the district role. In fact, this image is 
repudiated by research that shows that 
districts have played a significant role in a 
variety of state reform efforts (Fuhrman & 
Elmore, 1990). District roles in the face of 
increasingly active state policymaking have 
ranged from emphasizing particular pieces 
of state policies in order to further local 
goals (Firestone, 1989) to building on to the 
policy to increase its significance at the local 
level (Goertz, Massell, & Chun, 1998). 
Spillane (1996) has discussed how policy 
from a higher level of government 
frequently creates more policy at lower 
levels. This chapter sheds light on the extent 
to which the increase in local policy and 
activity is true in a variety of districts. 

Given the potentially broad range of 
district action that is possible in light of 
previous research, it is necessary to more 
clearly identify what I mean by “district 
response” to state accountability policy. In 
this area, my thinking was informed by 
research on districts that has regularly 
identified four particular areas related to 
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instructional improvement in which districts 
have tended to be active (Massell, 2000; 
McLaughlin & Talbert, 2003; Murphy & 
Hallinger, 1988). Those four areas are 
curriculum and instruction, professional 
development, data use, and resource 
allocation. As a result, my research focused 
on district strategies that related directly to 
any or all of these four functions. These four 
areas comprise the core functions that are 
theorized to lead to improvement of 
instruction and student performance. Key to 
their potential is the extent to which these 
functions are aligned and focused on a 
particular goal or set of goals. While later in 
this chapter I describe particular principles 
for assessing the quality of district initiatives 
in these areas, included among these 
principles is the degree to which functions in 
each area have the potential to support 
activities in the other areas. 

It is my hope that this research will 
contribute to our understanding of districts 
and their functions in several ways. First of 
all, much of the research on districts has 
studied districts that are, for some reason, 
“outliers.” Many recent studies of school 
districts have focused on those districts that 
are deemed to be particularly successful or 
engaged in unique and innovative practices. 
Researchers have based their identification 
of such districts on unexpectedly high 
student performance outcomes, or on the 
reputation of districts among researchers and 
practitioners (McLaughlin & Talbert, 2003; 
Snipes, Doolittle, & Herlihy, 2002; Togneri 
& Anderson, 2003). In either case, these 
“outlier” districts do not represent the bulk 
of school districts that are struggling with 
student performance and do not catch the 
eye or interest of well-known researchers 
and practitioners. This research attempts to 
rectify this knowledge gap by focusing on 
districts that have at least one high school 
that is performing below average on the 
state assessment, that have not, at the district 



level, demonstrated any dramatic 
performance gains (or losses) in the recent 
past, and that would be expected to be 
among those districts for which 
accountability pressure would be a 
particularly salient feature of their functional 
environment because of the presence of one 
or more low-performing high schools. The 
districts and individual high schools 
included in this study have been relatively 
flat in terms of performance on the 
standardized state exams. 

This research is also different from 
much of the previous research on school 
districts because it limits its focus to the 
work that central offices are doing with high 
schools. Much of the research that has been 
done on school districts, including much of 
that being built upon here, has examined 
how central offices interact with elementary 
schools. Until recently, that is where much 
of the districts’, and researchers’, attention 
has been focused. The limited attention that 
most districts have given to high school 
improvement, the significant difficulties of 
demonstrating improvement, and the 
increased pressure related to high school 
improvement make them a particularly rich 
area on which to focus district research. 

High schools have not demonstrated the 
achievement growth that has been seen at 
the lower levels of the K-12 system 
(Haycock & Huang, 2001; Olson, 2001). In 
part, this is due to the emphasis of lower 
grades within many accountability systems. 
Siskin (2003) has identified seven potential 
reasons why high schools pose more 
challenging contexts for change and 
improvement than do lower grades. Among 
the reasons Siskin cites is the fact that high 
schools are being asked to do something 
entirely new — have all students study and 
master a common set of standards in order to 
graduate. Additionally, she points out that 
high schools vary tremendously in terms of 
size, organization, performance, and 
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capacity. These differences, frequently 
within a single district, make designing 
improvement strategies particularly difficult. 
Such challenges were considered in studying 
the strategies that districts were using to 
stimulate high school improvement. 
Additionally, by limiting research about the 
effects of state accountability policies to the 
high school level, particular differences 
between state systems may become more 
stark. While all states are testing regularly in 
the elementary schools, the range of 
approaches to high school standards, 
assessments, and stakes is significantly 
broader. 

Additionally, much of the research on 
districts has examined one district function. 
Such studies include examinations of 
curriculum implementation, or professional 
development strategies, or data use. The 
research presented in this chapter looks 
across all four functions, with the conviction 
that the four functions are highly 
interrelated. A comprehensive study of 
districts’ strategies and responses requires 
an examination of the ways in which various 
strategies are aligned with one another and 
support a common goal. Finally, that 
research that does look at district behavior 
rarely considers in a detailed way the 
demands and impact of state policy. This 
study is focusing on those policies and the 
impact that they have on school district 
function. 

Conceptual Framework 

In considering the issues that I have 
mentioned and the previous research on 
district function, I have designed a 
conceptual framework (see Figure 1) which 
takes the state accountability policy as its 
starting point. Using the elements identified 
by Adams and Kirst (1999) to analyze state 
policy, I assess the mechanisms of influence 
that the policy uses to focus district 



attention. Additionally, I discuss district 
contextual characteristics and the ways in 
which they may serve to constrain or 
support district improvement. Both visually 
and metaphorically speaking, the district is 
“stuck in the middle” between these two sets 
of potentially competing, potentially 
complementary, legal and environmental 
demands and conditions. The ways in which 
districts deal with these messages from both 
sides and how the stimuli impact the 
practices that the districts select in carrying 
out improvement through the four functions 
identified earlier (curriculum and 
instruction, professional development, data 
use, and resource allocation) compose the 
bulk of this research. How are the district 
strategies serving to focus and motivate and 
build capacity for improvement at the high 
school level given their “middle” position? 
This map and my areas of focus attempt to 
capture the design of the state policy, the 
impact of that design on district practice, the 
meaning of district context for district 
behavior, and the overall approach that 
districts use in working with high schools to 
produce improved performance. 
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Figure 1 . Conceptual Framework Map 
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Research Design 

Because this research takes the state 
policy as its primary point of comparison, I 
began my research with two states — North 
Carolina and Pennsylvania — that have 
accountability policies that contrast on a 
number of levels. (These were two of the six 
states included in the larger CPRE study, the 
data from which are reviewed in this 
publication.) Typically, North Carolina and 
Pennsylvania are generally thought to be 
“strong” and “weak” accountability states 
respectively. They lie at opposite ends of an 
accountability scale developed by Camoy 
and Loeb (2004) that assesses the presence 
of particular elements such as school 
sanctions and rewards, high school level 
assessments, and high school exit exams. 
However, these states also differ in many of 
the ways in which they address the elements 
of the state accountability system identified 
by Adams and Kirst (1999). 

The two states identify different agents, 
the entities that are ultimately responsible 
for the changes that the state policy 
demands. North Carolina has identified 
schools as the level for change. As agents, 
schools receive sanctions and rewards as the 
agents. Pennsylvania has chosen to place 
responsibility for curriculum coverage, 
alignment, and improvement with districts, 
and sanctions districts for 
underperformance. Though data in 
Pennsylvania are publicly reported at the 
individual school level, and receives 
considerable attention from school and 
community, the state is holding the district 
responsible for improvement and has 
intervened in a number of school districts 
where more than half of the students are 
performing at a “Below Basic” level (the 
lowest of the state’s four performance 
categories). 

There is also a great difference in the 
ways in which the states authorize action 



and define accounts. North Carolina has 
standard courses of study for 1 1 courses at 
the high school level as well as end-of- 
course (EOC) exams to assess performance 
in each of these courses. Pennsylvania has 
one set of standards for high school students, 
to be met by the end of 1 1 th grade, and tests 
at that point to assess student mastery of 
these standards. (At the time of data 
collection for this research, standards had 
only been disseminated in math and 
language arts. Since then, additional 
curricular areas have been addressed.) 

These different standards and assessment 
strategies provide schools and districts with 
very different amounts and kinds of data. 
North Carolina’s data become available 
much more quickly following the end of a 
large number of courses, while 
Pennsylvania’s single high school 
assessment provides data (about high school 
students) that are generally not received by 
schools until the tested students are at the 
start of their 12 th - grade year. Table 1 
summarizes the differences between the 
states. 

In spite of the differences discussed 
earlier, there are similarities between the 
two state systems that are nearly universal in 
the United States. Both of the state systems 
are performance-based. They use changes in 
student performance on standardized 
measures as evidence of improved practice 
at the school level. These state systems are 
not seeking to monitor teacher practice or 
build capacity on any wide scale. They are 
focused entirely on outcomes. Related to 
this, both of these state systems, like most 
others, have been largely negligent in the 
area of ensuring causal responsibility. 
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Table 1 . Contrasting State Accountability Systems 





North Carolina 


Pennsylvania 


Identifying agents 


Schools 


Districts 


Authorizing action 


Course-level goals 


One set of high-school- 
level standards 


Managing agents’ 
productivity 


Performance-based 


Performance-based 


Defining accounts 


Detailed, frequent 


General, one-time only in 
high school 


Promoting compliance 


School-level sanctions and 
rewards 


District-level sanctions, 
school-level rewards 


Ensuring causal 
responsibility 


Minimal 


Minimal 



It is perhaps a conscious decision on the 
part of states to make clear the limited state 
involvement and to assure that districts and 
schools are responsible for increased 
achievement. However, there is little reason 
to assume that all schools or districts possess 
the capacity to engender the improvement in 
instruction that will be necessary to achieve 
the required targets. According to the 
theoretical functioning of these systems, 
such an assurance should be required before 
holding individuals or organizations 
accountable for their performance. Yet no 
state has developed the ability and resources 
to ensure such capacity at the local level. 
This is particularly true at the high school 
level, where continued debates about 
standards, the subject-specific nature of 
teacher expertise, and the lack of basic 
research about effective practices at the high 
school level make effective improvement 
strategies complex. 

To look at the range of district 
responses resulting from the policy choices 
of these two states, I used interview and 
document analysis. The data and analysis 
used in this chapter are a bit deeper than in 
some other chapters in this publication. By 



limiting my focus to only two states (out of 
six in the larger study), I was able to collect 
additional data during an iterative interview 
process and to engage in several rounds of 
analysis. Because accountability policy aims 
at eliciting individual responses, qualitative 
research is an appropriate method to assess 
those individualized situations and to look at 
practices that might be most appropriate in 
particular contexts. Analysis of this type of 
data can help to explain why appropriate or 
desired changes are, or are not, taking place. 
The data sources that were used for this 
research include interviews at the district 
and school level. Interviews were conducted 
during the 2002-2003 school year. One 
interview was done in each district prior to a 
site visit, and a complete set of interviews 
was done during a site visit using set 
protocols. The individual respondents that I 
selected at the central offices and the high 
schools were teachers and administrators in 
charge of the four functions that are at the 
heart of this work - curriculum and 
instruction, professional development, data 
use, and resource allocation. In addition to 
the staff members at the central office, I 
spoke to a sample of math and English 
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teachers and department chairs in each high 
school. Table 2 summarizes my sampling 
choices. 

Less formal follow-up interviews with 
particular district and school staff members 
were conducted where needed during the 
process of data analysis. I also collected and 
analyzed a set of documents, including state 
legislation about the policy, public 
information, surveys and self-studies 
conducted by the districts, and local and 
state reports related to achievement, 
demographics, and funding. 

Six districts in each of the two states 
were included in this study. After a 
preliminary analysis of data from all 12 
districts, I selected four districts (two in each 
state) to study in depth. My selection was 
based on the districts’ large size, providing 
at least the potential for regular interaction 
and intervention with the high school. I also 
sought demographic variation. In each state, 
one of the districts is wealthier, whiter, and 
more suburban, while the other is more 
urban, more heavily composed of students 
of color, and less wealthy. My sample was 
also purposive. It was selected after a 
preliminary review of data from six districts 
in each of the two states. I sought to 
represent the wide range of variation in 
response as we saw it within each state. In 
each of the districts, I collected data at the 
lowest performing high schools (where the 
district had more than one high school). This 



was done under the theory that if the state 
accountability pressure were causing the 
districts to respond by creating or 
implementing high school improvement 
strategies, district leaders would most likely 
target assistance toward the schools that are 
having the greatest difficulty demonstrating 
achievement within the state assessment 
system. Collection of data at the school level 
also provided a way to corroborate or 
triangulate data provided by central office 
respondents. Findings, however, are based 
on all 12 districts. From these, I chose 
exemplar cases to make the evidence most 
clear. 

I analyzed interview data using 
ATLAS .ti qualitative research software. 
Interviews were fully transcribed and the 
software allowed me to go through at least 
three rounds of coding for analysis. I relied 
on at least three types of coding as described 
by Strauss and Corbin (1990). Beginning 
with open coding, I was able to begin to 
label various phenomena in each of the 
districts. While I focused on the district 
functions and contexts that I had previously 
identified on the basis of research, this open 
coding does not preclude the identification 
and labeling of previously unconsidered 
action. For me, open codes consisted of 
broad categories like “professional 
development,” “data use,” “history,” and 
“policy knowledge.” 



Table 2. Sampling Choices 



Districts 

(4 in-depth, 12 total) 


High Schools 

(5 in-depth, 12 total) 


Individuals (63) 


• large size 


• lowest performing 


• administrators, math 


• demographic variation 




and ELA teachers 


• representative 




• central offices (25) 


variation in response 




• high schools (38) 
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During this process, I simultaneously 
began to consider patterns that exist and to 
look for commonalties and differences 
within and between the cases. Axial coding, 
which involved not only the district and 
school interviews but the state and district 
documents as well, helped me to identify 
potential interactions. It was through the use 
of axial coding that theory began to emerge 
about the particular interactions occurring 
between states and districts. Axial coding 
included both potential causative 
relationships like “response to state” or 
“superintendent goal,” as well as 
assessments of action like “targeted,” “high 
quality,” and “passive.” Finally, using 
selective coding, I produced the narrative 
account that allowed me to generate case 
studies of each district as well as to begin to 
understand the influence that state 
accountability policies are having on school 
districts in both states. This final stage of 
coding was done by identifying particularly 
important relationships and quotations that 
could provide evidence supporting my 
findings. 

Once case studies were generated for 
each of the four districts, I looked across the 
four areas of district function and attempted 
to describe and assess the manner in which 
district functions are being introduced, 
shared, and supported. I have created five 
continua to capture this approach. Within 
each range, the first descriptor represents 
behavior more likely to result in 
improvements in performance at the high 
school level that are aligned with the 
demands of the state accountability system. 
The latter descriptor represents a less 
strategic approach by the districts to high 
school improvement. These continua are 
based on a theory of systemic reform that 
demands a coordinated and active response 
to shifts in the policy environment (Smith & 
O’Day, 1991). 



• Interventionist to noninterventionist. 
Interventionist central offices staff are a 
regular presence in the high schools in 
the districts. They are very “hands-on” 
with regard to the operation of high 
schools, using school staff to support 
centrally defined programs and goals. A 
noninterventionist central office may 
possess expertise equal to its 
interventionist counterpart but may 
function much more as a support 
provider, offering assistance only when 
requested by school personnel. 

• Active to passive. Central offices 
classified as active are regularly 
attempting to identify both challenges 
and solutions for their high schools. 
They are engaged in meaningful 
searches in advance of school requests. 
An active district is not necessarily 
interventionist. For example, a district 
that is active in a particular area may 
offer a host of carefully selected 
professional development programs, 
though it may not require anyone in 
particular to attend. In this way, the 
district can be actively working to help 
high schools but noninterventionist in its 
approach. Districts may be active or 
passive in one particular functional area, 
or all four. 

• Differentiated to uniform/generic 
assistance. The approach that a district 
takes with regard to a high school may 
be extremely individualized in seeking 
to meet the unique needs of a particular 
school. This implies that the central 
office has developed a unique 
understanding of each school and has 
developed a plan to improve the school. 
In contrast, a district may have a more 
uniform approach to school 
improvement and will seek to meet 
needs that the central office has 
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identified as being universal. In an 
optimal case where central office 
assistance is differentiated, it may be 
differentiated by high school 
performance level or by grade level. It 
may be differentiated within a particular 
function (for example, the district may 
provide professional development in 
each school though the various 
approaches may be very different) or 
across functions (meaning that the 
functions in which the district is active 
with regard to each school may be very 
different). 

• Prescriptive to nonprescriptive. Within 
any of the functions identified earlier, 
the district may be extremely 
prescriptive in the changes or initiatives 
that schools are to undertake, meaning 
that they determine the improvement 
approach for the schools. On the other 
end, the district may determine the need 
for improvement in an area, but may 
then offer schools a range of options or 
allow them to seek out their own 
remedies. While I have placed the 
prescriptive identifier at the improved 
end of the continuum, there is 
considerable debate about the 
appropriate locus of decision making. 
However, because I have targeted 
schools with some history of low 
performance in this research, it is likely 
that school leaders do not possess the 
capacity needed to improve performance 
(if they did, their schools would not be 
low-performing). For this reason, the 



locus of decision making should, at least 
temporarily, be removed from the school 
itself. 

• Coherent to unaligned. This continuum 
cuts across the areas of district function 
to describe how well the set of 
improvement efforts in a district is 
arranged with regard to a coordinated 
focus for teachers. It refers to the ways 
in which the district-initiated 
improvement efforts in the four areas 
work together to create a coherent 
system that is arranged around a clear set 
of goals and priorities. 

Describing district approaches along 
these dimensions will serve to paint a fuller 
picture of the role the district is playing. I 
created this system of research-based 
qualities in order to describe and analyze the 
approach to high school improvement that 
districts are pursuing. In order to categorize 
districts efficiently in subsequent discussion, 
I refer to those districts with behaviors that 
represent the improved end of the spectra 
described (that is, interventionist, active, 
differentiated, prescriptive, and coherent) as 
“Type A” districts. Those districts whose 
behaviors are closer to the unimproved end 
of the spectra (noninterventionist, passive, 
uniform, nonprescriptive, unaligned) are 
labeled “Type B” districts. It was the case 
that districts tended to fall into one category 
or the other, as shown in Table 3. For 
example, districts were unlikely to be on one 
end of certain spectra while on other end of 
other spectra. 
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Table 3. District Types and Descriptors 



Type A School District 


Type B School District 


Interventionist 


N oninterventionist 


Active 


Passive 


Differentiated 


Generic 


Prescriptive 


Nonprescriptive 


Coherent 


Unaligned 



However, it is theoretically possible that 
Type A districts were prescribing and 
supporting high school interventions that 
were not likely to lead to improvements in 
student performance. Optimally, districts 
would engage in high-quality initiatives in 
each of the four functional areas while 
selecting and supporting them in ways 
represented by the improved ends of the 
continua. 

In order to assess the quality of the 
initiatives (as opposed to the general 
approaches described by Type A or B) of the 
districts’ actions, I compared the particular 
content of the districts’ responses to “best 
practice” in each of these areas as they have 
been widely endorsed by prominent 
scholars. Relying on a range of research in 
the areas of professional development, data 
use, and resource allocation, I was able to 
make judgments about the quality of the 
practices that districts were using or 
encouraging their high schools to use. I did 
not develop a continuum on the fourth 
function studied, curriculum and instruction, 
because it lacks professional consistency on 
what qualifies as improved practice. The 
debates about the merits of direct instruction 
versus constructivist or discovery learning 
approaches are just one example of the 
diversity of professional opinions that exist. 
Such an inconclusive environment prevents 
me from evaluating the curriculum and 
instruction choices made by a school 



district. Table 4 provides an example of the 
quality measures that were used and the 
ratings given to two sample districts, one of 
each “type.” (The ratings of the sample 
districts here were based on an analysis of 
the programs in place in each of the 
districts.) 



108 




Table 4. Qualities of District Response 



Principles of successful professional development 2 


Type A 


Type B 


Content focuses on what students are to leant and on addressing different 
challenges students may have 


Moderate 


Weak 


Based on analyses of differences between performance and goals 


Strong 


Weak 


Involves teachers in the identification of what they need to learn and the 
development of learning experiences 


Moderate 


Moderate 


Primarily school-based and built into the day-to-day work of teaching 


Moderate 


Weak 


Organized around collaborative problem solving 


Weak 


Weak 


Continuous and ongoing, involving follow-up support and external sources 


Strong 


Weak 


Incorporates evaluation of multiple outcome measures 


Strong 


Weak 


Includes an opportunity to gain an understanding of the theory underlying the skills 


Weak 


Weak 


Connected to a comprehensive change process focused on improving student 
learning 


Strong 


Weak 



Principles of effective data use 1 ’ 


Type A 


Type B 


Focus attention on performance indicators directly relevant to teaching and learning 


Strong 


Moderate 


Motivate staff to attend to relevant data 


Moderate 


Moderate 


Develop knowledge of school staff about how to use data to take action 


Moderate 


Weak 


Use data in order to make decisions about resource allocation 


Strong 


Weak 


Use individual level student data to better meet student needs 


Strong 


Moderate 


Generate additional data as needed to improve program and practices 


Moderate 


Moderate 



Principles of effective resource allocation 0 


Type A 


Type B 


Reduction of specialized programs 


Moderate 


Weak 


More flexible student grouping 


Moderate 


Moderate 


Structures to support more personal relationships 


Moderate 


Weak 


Longer and more varied blocks of instructional time 


Moderate 


Moderate 


More common planning time 


Moderate 


Weak 


Increased support for curriculum review and improvement 


Strong 


Weak 


Support for high-quality professional development opportunities 


Strong 


Weak 


Resources for improved data analysis and use 


Strong 


Weak 



a These principles are drawn from the work of the National Partnership for Excellence and Accountability 
in Teaching (2002). b These principles are a combination of concepts described in an article written by 
Jennifer O’Day (2002) and ideas emerging from the data. c These principles were identified on the basis of 
the work of Miles & Darling-Hammond (1997). 

Findings 

As is evident from Table 4, Type A 
districts tended to employ practices that 
were supported by research, while Type B 
districts were much less invested in 
research-based practice. In many ways, this 
is encouraging news. Districts that were 
more highly involved in their high schools 
were promoting good practice, while those 
that were more passive had a less well- 
designed set of improvement strategies. 



Among the six districts in North Carolina, 
there was an even split between Type A and 
Type B districts. In Pennsylvania, there were 
four Type A districts and two Type B 
districts. Though districts had been selected 
because of the presence of a low-performing 
high school, district behaviors or potential 
for improvement were not factors in the 
original selection. With such a small sample 
size, it is not appropriate to make universal 
judgments about the proportion of districts 
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of each type in each of the states. However, 
two points should be made. First, despite 
Pennsylvania’s less well-developed 
accountability system, there was a greater 
number of Type A districts in that state than 
in North Carolina. For reasons that will be 
discussed, while the numbers of districts of 
each type in each state were similar, the 
patterns in district response were much more 
clear in North Carolina than in 
Pennsylvania, where district response tended 
to be less predictable and more 
idiosyncratic. Second, this study included 
only those districts with high schools that 
are struggling. It is quite possible that if the 
sample were more random, and included 
districts with more well-performing high 
schools, the proportion of Type A districts 
might in fact be significantly greater. 

Type A Districts 

One of the most salient characteristics 
of Type A districts was that respondents 
described making a change in district 
practice in response to state pressure. There 
was evidence that the district had changed as 
a result of state policy initiatives. Though I 
was dependent upon the retrospective view 
of respondents for measurement of change 
over time, there were some very telling 
statements by district leaders that gave 
insight into the question of whether districts 
had changed their approaches since the 
introduction of state accountability systems 
for high schools. One central office 
administrator said in a North Carolina Type 
A district said: 

I guess a lot of my role has changed from 
just when 1 first got the job it was, OK, 
does everybody have that they need? And 
we’ll provide some staff development 
every year too. I’ve had to be the change 
agent to push them to get things going. 
I’ve had principals that frankly have been 



in position so long that everything status 
quo is fine with these folks, I’ve had to 
drag them along. 

It is evident that there has been change in 
the district and that much of it is emanating 
from the central office. Similarly, when 
asked how practices in his district had 
changed since the introduction of the latest 
version of the state accountability system, a 
superintendent in one of the Type A districts 
in Pennsylvania said: 

I’m much more autocratic. 1 did all the 
site-based and all the team and we all 
hugged each other and everybody was 
happy and the scores were going down, 
literally through the basement. . . . Now, I 
will entertain any idea if they’ve done a 
review of literature, come with a 
proposal, show me the accountability, 
show me the benchmarks. So from that 
sense I think I’ve become less tolerant. 

Type A district leaders clearly had 
approaches that had been affected by state 
policy. 

In Type A districts, central office staff 
is very interventionist. It is clear from the 
previous quotation that district staff is 
involved in the selection of strategies at the 
school sites. Staff from Type A districts are 
out in the schools, working in a very hands- 
on way with schools, not waiting for school 
staff to come to them with questions or 
problems. They are actively looking to 
identify problems in schools, as well as 
actively looking for solutions. District 
administrators in Type A districts were 
frequently visiting classrooms and actively 
engaged in examining performance data in 
order to help teachers and administrators in 
their high schools to identify challenges and 
select resources. The searches that they led 
often represented the more in-depth and 
successful strategies as represented by 
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Gross, Kirst, Holland, & Luschei (in this 
publication). 

These districts also have a differentiated 
approach to their work with high schools. 
They are getting to know each individual 
high school (where there is more than one) 
well enough to be able to assess strengths 
and weaknesses. The districts select 
strategies needed for improvement in 
particular schools. For example, in one Type 
A district in North Carolina, the central 
office had done an assessment and identified 
the challenge in one of its high schools as 
being an issue of teacher skill. In another 
high school in that same district, teachers 
demonstrated sufficient teaching skill on the 
individual level but the school lacked any 
sense of community or collaboration, in part 
because of a lack of leadership at the 
building level. These two “diagnoses” 
required and received very different 
remedies, ranging from professional 
development in the former to a change of 
leadership in the latter. While districts 
maintain a differentiated approach to 
working with high schools, it is not a site- 
based approach to differentiation. 

Once Type A districts have done an 
assessment and diagnosed a problem, they 
tend to be rather prescriptive. They direct 
their high schools to implement certain 
practices, and monitor the implementation of 
those practices. In some cases, the high 
school staffs felt that they had made the 
selection of a certain practice. In 
conversation with district leaders however, it 
became evident that high schools were in 
fact presented with some very limited 
choices, all of which the central office had 
already approved. 

Finally, Type A districts possess a 
coherent strategy for improvement of 
student performance. They are aligning their 
practices and goals around a target. Most 
commonly that target is improved student 
performance on the state assessment. The 



districts have adopted that as their goal, 
occasionally going beyond the state target, 
and are working to align a range of practices 
that they believe will lead to improvement. 
These behaviors differed in the two states in 
accordance with the design of the state 
assessment system. However, regardless of 
the state, Type A districts are doing 
assessments of instruction, measurements of 
learning and improvement, and making 
decisions about what is working and what is 
not. In Type B districts, much of this 
responsibility stays at the high school level. 

Type B Districts 

Type B districts represent a very 
different approach. These central offices are 
much more passive. They view themselves 
as one of a variety of external resources 
available to schools. They wait for schools 
to ask for help, and then are available as a 
resource. In contrast to the Type A districts, 
they have not changed practice as a result of 
the demands of state accountability. The 
superintendent in one North Carolina Type 
B district said, “When I look back at the big 
picture of what we do and how we do it, I 
think we operate somewhat similar to the 
way we have been operating for the last 10 
years.” Given the fact that the North 
Carolina state accountability systems at the 
high school level only began 6 years prior to 
my data collection, it is clear that there has 
been a minimum of improvement-oriented 
response in this district and those like them. 
In Pennsylvania, where the state 
accountability policy for high schools has 
been in place for a shorter time than in 
North Carolina, a superintendent in a Type 
B district said that he could not point to any 
changes that had been made in district 
practice in the last 3 to 5 years. Given that 
the current state accountability system had 
only been in place for 3 years at the time of 
data collection, it became clear that this 
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district had continued to function relatively 
unchanged in spite of the change to state 
policy. Neither district was alone in their 
respective states in terms of their static 
practice. 

When Type B districts do act, they tend 
to adopt programs without clear matches to 
school needs and rarely follow up to see that 
the programs are being implemented as 
planned. They frequently do not have a clear 
goal around which they are aligning efforts. 
While Type A districts most frequently cited 
improvement in test performance as their 
central goal. Type B districts cited 
everything from character development to 
technology enhancement to improvement in 
state test performance. This wide range of 
goals caused effort to be refracted in a 
number of directions. 

Districts in both North Carolina and 
Pennsylvania tended to fall into the 
typologies represented here by Type A and 
Type B. Their overall approaches to high 
school improvement, and the quality of the 
strategies that they were using tended to be 
very similar, in accordance with the group 
into which they fell (Type A or B). 

However, districts in the two states tended to 
focus on slightly different areas of activity. 
In Pennsylvania, Type A districts were 
frequently focusing on things like creating 
explicit grade and course level standards. 
They were just beginning to think about 
grade-level articulation and pacing guides to 
help teachers move through the district 
curriculum at a speed that would allow 
coverage of all of the standards. 
Pennsylvania Type A districts were heavily 
focused on introducing teachers to the state 
standards and reinforcing the connections 
between standards, curriculum, and 
assessments. In North Carolina, this was not 
necessary, as state-defined course level 
standards and pacing guides are an accepted 
part of teaching in most school districts. 
Instead, districts in North Carolina were 



analyzing the more frequent, detailed, and 
course- specific data that the state system 
there provides and were trying to take action 
related to that data. So, while districts 
broadly fell into the Type A and B 
categories, the particular improvement 
strategies that they were using varied 
widely. 

Analysis 

While the district types in both states 
were similar, across the six districts in each 
state it was much easier in North Carolina 
than in Pennsylvania to predict from district 
characteristics which districts would behave 
in a Type A or B fashion. In North Carolina, 
the determination could be made from the 
confluence of three factors: previous 
performance, central office size, and 
leadership. Those districts that had 
particularly low-performing high schools 
were more likely to behave in a Type A 
manner. This makes sense given that the 
lowest performing schools are most in 
danger of state intervention. However, in 
order to respond in the way described 
previously, central offices had to have 
sufficient numbers of staff to work with high 
schools on an ongoing basis. In a district 
that did have one very low-performing high 
school, the small size of the central office 
made it impossible for staff to work with the 
high schools on a regular and interventionist 
basis. In North Carolina, Type A districts 
tended to have an average of more than 
twice as many professional staff leaders at 
the district level in comparison with Type B 
districts (29 in Type A districts versus 13 in 
Type B districts). Finally, in each of the 
Type A districts there were one to three key 
leaders who strongly believed that it is the 
role of the central office to direct schools 
and not simply to serve as an additional 
resource. This team of leaders had a very 
clear and uniform approach and set of 
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strategies that they were pushing high 
schools to use. It is perhaps somewhat 
counterintuitive that districts with the lowest 
performing high schools had the most 
promising practices. The fact is that in most 
cases, the overall district approaches and 
particular strategies in the Type A districts 
were relatively recent innovations. We do 
not yet have sufficient time to measure the 
impact of their instructional improvement 
efforts on student performance. 

In Pennsylvania, it was not possible to 
predict from a particular set of district 
characteristics which districts were likely to 
be behaving in a Type A or B fashion. As in 
North Carolina, issues of staff size, previous 
performance, and district leadership were 
certainly important. However, they were not 
as determinant in Pennsylvania as they were 
in North Carolina. While in North Carolina 
all of the Type A districts had certain 
contextual characteristics in common, this 
was not the case in Pennsylvania. Type A 
districts in Pennsylvania were both large and 
small and had high schools that ranged 
widely in terms of achievement. Response in 
Pennsylvania districts, it seems, depended to 
a much greater extent on the individual 
entrepreneurship of local actors. In 
Pennsylvania districts, the primary 
determinant was the presence of what has 
been called a “dominant coalition” 
(Firestone, 1989). In Type A districts in 
Pennsylvania, changes were dependent on a 
group of individuals (most frequently 
district and high school administrators) who 
decided to make use of the state system in 
order to push forward an improvement 
agenda that would support both the state 
system as well as their own goals. 

The differences between district 
response patterns raise the question of why 
response in North Carolina was so much 
more predictable on the basis of certain 
contextual characteristics, than was response 
in Pennsylvania. In North Carolina, it 



appears that those districts that contained a 
high school that was a potential target of 
sanction under the state accountability 
system were very likely to respond, 
provided they possessed the staff capacity to 
do so in an effective manner. In 
Pennsylvania, even those districts with high 
schools that were performing at very low 
levels did not necessarily feel the same 
pressure to devise a strong response or 
improvement plan pegged to the state 
system. Later, I will discuss some reasons 
why districts reacted or failed to react. 

Analysis of the data made clear five 
policy factors that differ under these two 
systems and help to explain the different 
response patterns in the two states. In 
addition to the conscious policy choices 
described by the Adams and Kirst (1999) 
framework, the reasons for the contrast 
between the states cut across the six 
elements of accountability policies described 
earlier (identifying agents, managing 
productivity, defining accounts, etc.). These 
policy features affect the way in which the 
overall message as well as particular pieces 
of information impact districts. These 
characteristics of state policy design and 
implementation that impact district response 
patterns are what I label the “Five C’s.” 
While the stakes associated with the state 
accountability systems were considered in 
the sampling frame for the overall study, the 
characteristics identified in the following are 
independent of the level of stakes, but rather 
describe policy design using a different 
frame. 

Five C's 

Consistency 

North Carolina’s system is internally 
consistent. Schools are treated as agents 
throughout. They are subject to all of the 
requirements, rewards, and sanctions that 
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the state has to offer. They are also the 
targets of change. The state wants the 
learning process in the schools to change in 
order to align with state standards. In this 
system, all of the mechanisms in the system 
are focused on the unit that is expected to 
change. In Pennsylvania, districts are agents 
for setting local standards, accounting for 
performance, and being sanctioned. But 
schools are agents when it comes to 
receiving rewards and are the most frequent 
subjects of public performance reports. The 
state has designated districts as the primary 
agents but is also looking for change at the 
school level. Thus, it has designed a system 
that presses for action at one level in the 
hopes that it will influence practice at 
another level. The state accounts for 
performance not at the level of the primary 
agent, but at a subordinate level. There is an 
inconsistency in the logic of the system. It is 
not surprising then, that a more consistent 
system, like the one in North Carolina, 
would have more consistent responses at all 
levels of the system. 

Credibility 

North Carolina has accrued a much 
greater degree of credibility for its goals and 
standards than has Pennsylvania. The vast 
majority of people with whom we spoke in 
North Carolina felt that the standard courses 
of study included information that students 
completing a particular course should 
master. The goal of proficiency in a course, 
as demonstrated by the EOC exams, seemed 
to matter to all staff. In Pennsylvania, it was 
much more common to hear school and 
district staffs question both the origins and 
importance of particular state standards as 
well as the reliability and validity of the test 
being used to measure mastery of those 
standards. The system used to both authorize 
action and define accounts lacked legitimacy 
in the eyes of many respondents. 



Additionally, respondents in North Carolina 
were more likely to agree that the required 
assessments were reasonable for all high 
school students. In Pennsylvania, some 
respondents questioned the need for all 
students to meet certain standards. This was 
particularly true in math, where the 
necessity of high levels of math skills was 
questioned. As a result of this, school staffs 
were more amenable in North Carolina to 
learning from their districts about how to 
help students to excel on state exams. In this 
situation, performance becomes a (nearly) 
universally valued outcome, making it easier 
for intermediary organizations to assist with 
its attainment. In Pennsylvania, school and 
district staffs were less likely to embrace the 
state standards and testing, adding another 
layer of resistance to change. 

Chronology 

It is very likely that part of North 
Carolina’s credibility has been achieved 
simply with the passage of time. In North 
Carolina, the state accountability system had 
been in place without any obvious 
significant changes for 6 years at the time of 
my visits. In Pennsylvania, the state policy 
had been in place for just 2 years, and during 
that 2-year period had seen changes 
regarding the highly visible question of 
placing endorsements based on test 
performance on student diplomas. District 
leaders in North Carolina expressed 
familiarity with the state accountability 
system and confidence that it would not be 
eliminated. In contrast, many teachers and 
leaders in Pennsylvania did not feel certain 
that the state accountability system would 
last through the new governor’s 
administration. The different impact that 
these two attitudes have on organizational 
behavior at the district level cannot be 
underestimated. Though there is no real way 
for the state to address this at the outset, it 
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does speak to the need for states to commit 
to a system over the long term, rather than 
continuously making changes that may 
undermine agent confidence. 

Comprehensiveness 

An accountability system that provides 
a full set of standards at the high school 
level as well as data in sufficient depth and 
detail to assess the attainment of those 
standards is much more likely to have a 
systematic effect on districts than is a 
system that is weak in both of these areas. 
Data about performance is the essential tool 
of the accountability system. It must be 
tightly linked to, and aid in the instruction 
of, clear standards. Without rich standards 
and data from the state, districts in 
Pennsylvania that want to focus high schools 
on making instructional change must first set 
the standards and create the measures. The 
process of clarifying standards and creating 
(or choosing) appropriate assessments is a 
lengthy one that not only consumes 
significant time and resources, but further 
confuses the role of district as principal or 
agent within the system. The variation that 
occurs when the content and progression of 
courses is left at the local level makes it 
more difficult for groups of districts to 
collaborate with each other and with outside 
entities on improving both instruction and 
data analysis. Leaders in Type A districts in 
North Carolina were much more likely to 
report collaborating with other districts, 
regional consortia, or even the state, because 
all units of the system shares clear, statewide 
goals. Accountability is all about setting 
clear goals and having a clear idea about 
how the agents are progressing toward these 
goals. Without such clarity, there is more 
“noise” in the system that can allow districts 
to become distracted and consumed with 
actions that will not necessarily lead toward 
mastery of state goals. 



Consider the Type A districts discussed 
in this chapter. One in North Carolina is 
much more focused on getting all students to 
proficient levels on the EOC exams. It is the 
guiding mission for the district and provides 
their activities with a coherence they might 
otherwise lack. Another in Pennsylvania, 
while also Type A and working to improve 
high schools, is creating particular course 
sequences for each of its high schools, 
manipulating programs to attract certain 
groups of students, and preparing teachers 
with good instructional practice. They are 
just beginning to consider making 
curriculum consistent across the district and 
linking it to standards. The comprehensive 
nature of the system in North Carolina 
means that those districts that are 
predisposed to intervene in high schools 
(due to their size and capacity, risk of being 
sanctioned, and leadership) will be more 
likely to act in a way that is directly aligned 
with the state system. In Pennsylvania, 
districts that may possess similar 
characteristics may act in a variety of ways 
that may or may not be aligned with the 
state system. 

Comprehensibility 

This characteristic attempts to describe 
how well district (and to a lesser extent, 
school) staffs understand all of the aspects 
of the state system that may impact them. 
Respondents in North Carolina had clearer 
understanding of the state’s accountability 
system than did respondents in 
Pennsylvania. While respondents in both 
states were not entirely clear on how certain 
algorithms related to annual growth are 
calculated, North Carolinians overall were 
much more aware of the categories that their 
high schools fell in and what would be 
required to receive a performance bonus. 
They were also more familiar with the 
consequences of failure, namely the 



115 




presence of state assistance teams. 
Pennsylvanians expressed confusion about 
how performance bonuses were decided, 
shared a wealth of rumors about how state 
standards were set, and did not seem clear 
on what state intervention might mean 
should it occur. This confusion extended to 
student-level stakes as well. Several staff 
members told me that they were not certain 
what role the Pennsylvania System of 
School Assessment (PSSA) played in high 
school graduation. 

Because the theory of accountability is 
at its heart an effort to motivate agents to 
comply with the demands of principals, it is 
extremely important that agents understand 
both the demands as well as the 
consequences of success or failure in 
meeting those demands. One of the reasons 
for greater predictability about the impact of 
the policy on school districts in North 
Carolina is the more uniform level of 



understanding about the policy among 
district and school staffs across the state. In 
Pennsylvania, where the understanding of 
the system was much more variable, and 
poorer overall, it is more difficult to predict 
how districts must react. One Pennsylvania 
district, for example, has had what might be 
considered a “North-Carolina-like” response 
to the state policy. It also has a 
superintendent with an administrative 
connection to the state board of education 
and has a very rich understanding of state 
policy. He has shared that information with 
a set of colleagues with whom he has 
worked for a number of years. Without such 
a complete picture, we may not have seen 
the same response. Another Pennsylvania 
district, with larger percentages of students 
in the lowest performing category on the 
PSSA, has not demonstrated such a coherent 
or interventionist approach. 



Table 5. Review of Distinguishing Policy Features 





North Carolina 


Pennsylvania 


Consistency 


Schools get sanctions and 
rewards 


Districts get sanctions, schools get 
rewards 


Credibility 


Standards and assessments are 
reasonable 


Tainted by politics, of questionable 
validity and utility for all 


Chronology 


In place since 1996, stable 


In place since 2000, with changes 


Comprehensiveness 


1 1 course standards and 
assessments 


One set of standards, one assessment 


Comprehensibility 


Provides growth and status 
categories 


Provides only overall averages 
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Discussion 

I want to be clear that all districts, in 
both states, were responding to the state 
accountability system to some degree. They 
were aware of the state standards and were 
making some effort, be it ever so modest in 
some cases, to align curriculum with 
standards and to provide at least some 
professional development related to the 
standards. However, as discussed 
previously, in Type A districts, response was 
generally aligned with professional opinion 
about instructional improvement. These 
districts were using approaches that were 
closer to what scholars indicate is “best 
practice.” Type B districts, in addition to 
being less active and interventionist, were 
employing practices that were less 
supportable with existing scholarship. It 
became clear that the match between state 
policy and district context was particularly 
salient in making a district Type A or Type 
B. State policies only “spoke to” or were 
being “heard by” certain subsets of districts, 
generally those that I have classified as Type 
A. Some districts were not able to respond 
(e.g., too small or lacking leadership), while 
others did not feel that the policy affected 
them (e.g., performing close to average). 

For Type A districts, the theory of 
action that supports performance-based 
accountability was working. The policy 
spoke to districts and the districts had the 
ability to act in ways that would lead to 
improvement in areas that the state was 
measuring. It is perhaps more important to 
ask why some districts were not responding 
to the state accountability system, despite 
the presence of at least one high school that 
was performing below the state average. 
There are two possible explanations for this. 
It is possible that there was something 
lacking in the policy design that did not 
consider local contextual factors. For 
example, if the low-performing high schools 



in a district were closer to the state average, 
the district was less likely to be proactively 
engaging its high schools in improvement 
efforts. Or if districts were very small, they 
frequently did not possess the capacity to 
engage in the sorts of high school capacity 
building that larger districts were able to do. 
Alternatively, the policy design did not 
convey to these districts the press or 
direction that designers may have hoped. 
The reasons for this had to do with the Five 
C’s. Features of policy design and 
implementation combined to send mixed or 
unclear messages to actors at the district 
level. 

It appears that where state 
accountability policy does meet the five 
criteria discussed previously, it is possible 
for policy to influence a distinct and 
predictable subset of districts. On the basis 
of the sample of six districts examined in 
this study, this is the case in North Carolina. 
Given particular district contextual factors 
(size, performance, leadership), it is 
relatively easy to predict whether a district 
will be Type A or Type B. Where the policy 
is not clear, response is much more 
dependent on local factors. Such a situation 
creates the kind of unpredictable and 
idiosyncratic variation that we saw in 
Pennsylvania. However, as we saw, this did 
not lead to an absence of supportive and 
proactive districts. In fact, in this small 
sample, Pennsylvania had a slightly higher 
proportion of such districts than did North 
Carolina. However, the focus of this 
research is the consistency with which 
districts can be moved to Type A behavior. 
There is no doubt that the policy in North 
Carolina moves districts in a more 
predictable and systematic way than does 
the policy in Pennsylvania. 

The challenge that this research poses 
for policymakers who wish to create 
successful and predictable patterns of 
response is at least twofold. First, policy 
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created with attention to the Five C’s will be 
more likely to send a clear message to a 
distinct group of local actors and to elicit a 
response. However, as we saw in North 
Carolina, even a well-constructed policy 
does not provoke action by all of the school 
districts that need improvement. This raises 
the issue of whether one can construct a 
policy that is both clear enough and broad 
enough to turn Type B districts into Type A 
districts. Such a policy would need to 
address both motivation issues — in order to 
get the attention of more districts, rather 
than be ignored as it was in many Type B 
districts — as well as local context and 
capacity issues. Some districts could not 
respond to state policy because they lacked 
sufficient staff or leaders who knew about 
appropriate improvement strategies. Without 
outside assistance, these districts cannot 
become Type A’s regardless of the how 
much they might want to, on the basis of the 
motivation the state policy provides. 
Accountability policies may not be 
sufficient to address all of the needs of a 
Type B district, but other state programs 
should consider these issues if the state is 
seeking to maximize the benefit to be gained 
from local school districts. 

Future Research 

This research was based on a relatively 
small sample of districts, six in each of two 
states. It found relatively similar proportions 
of Type A and Type B districts in each state. 
It would be worth expanding the sample 
sizes to see if the proportions remain similar. 
Also as part of this expansion, it would be 
possible to see if the predicting factors in 
North Carolina remain the same and whether 
any patterns in district response in 
Pennsylvania become apparent. This work 
would help to elucidate some of the key 
contextual factors that are either allowing 
districts to become Type A or are preventing 



them from doing so. These are the context 
issues at the district level that will need 
policy attention and support if districts are to 
be a viable resource in encouraging and 
helping high schools to improve. 
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CHAPTER 6 
Summary 

Margaret E. Goertz and Diane Massell 



Introduction 

American public education faces 
increasing pressure to carry out its 
mission of preparing youths with the 
skills to compete in today’s global 
economy and to participate 
constructively in a democratic society. 
As part of this pressure, policymakers 
have developed increasingly 
sophisticated accountability and support 
systems in efforts to steer schools 
towards improved performance. These 
“new accountability” approaches 
emphasize student performance over 
system inputs, focus on schools rather 
than school districts as units of 
improvement, and use public reporting 
of student outcomes and rewards and 
sanctions as ways to motivate schools to 
alter their curriculum and instructional 
practices (Fuhrman, 1999). These 
strategies embody two key assumptions: 
(a) that accountability systems can be 
made powerful enough to influence the 
behavior of schools and (b) that schools 
have or will develop the capacity to 
identify, select, and implement policies 
and practices that will improve their 
performance. 

State and national assessment 
results show that many elementary 
schools have grown in educational 
performance over the last decade. Some 
researchers have argued that a portion of 
these gains can be attributed to state 
accountability systems that have set 
standards, focused attention, and created 



incentives for improved performance 
(Camoy & Loeb, 2004; Grissmer & 
Flanagan, 1998; Grissmer, Flanagan, 
Kawata, & Williamson, 2000; Hanushek 
& Raymond, 2002). High schools, 
however, have not experienced the same 
positive effects, and we know little about 
how high schools respond to external 
accountability pressures. The preceding 
chapters provided insight into how 
teachers and administrators in American 
public high schools are influenced by 
and attempt to address the problems 
posed by the new accountability. Our 
analysis of 48 high schools in six states 
builds upon earlier studies with smaller 
and less representative samples of 
secondary schools, sometimes agreeing 
with and sometimes challenging their 
conclusions about accountability. In this 
final chapter, we review several of our 
key points, and discuss their implications 
for policymakers. 

Accountability— A 
Stimulus for Action 

This first phase of our study 
confirmed the point made by others that 
state accountability systems can focus 
educators on reform, and motivate them 
to address content standards and 
measured student performance. State 
accountability policies clearly shaped the 
goals and many of the challenges that 
high school staff identified, and 
influenced the actions that they 
undertook. To be sure, educators had 
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goals and initiatives not related to 
accountability. For instance, teachers 
and administrators were concerned about 
keeping students in school through 
graduation, student success in 
postsecondary education, student 
motivation and social/emotional needs, 
and academics beyond those measured 
by state tests. But schools, departments, 
or individual teachers also adopted a 
plethora of accountability-related 
initiatives, from voluntary tutoring 
sessions or test preparation activities to 
more comprehensive overhauls of 
curriculum and instruction. 

In general, we found greater levels 
of response in California, Florida, New 
York, and North Carolina, states whose 
strong accountability systems had major 
consequences for both students and high 
schools. Intriguingly, however, we also 
found substantial responses from many 
high schools in Pennsylvania and some 
in Michigan, although these states had 
no sanctions for poorly performing 
students or schools at the time of our 
study. Indeed, state accountability drew 
attention even when educators were not 
held directly responsible for student 
performance. For instance, although high 
school teachers did not think their job 
was at risk if students earned poor test 
results, professional pride and concern 
about their students, their administrators, 
and/or the reputation of their school 
motivated many to address aspects of 
accountability. 

Furthermore, we were also surprised 
to discover high levels of response even 
when educators expressed strong 
criticisms of their state accountability 
systems, a finding which in some ways 
contradicts previous implementation 
literatures. Educators in many of our 
schools questioned the operational and 
technical aspects of their state’s testing 



and accountability programs, such as the 
infrequency of test results, late reporting, 
the way progress was measured, whether 
expectations for students were 
reasonable, and the like. They also 
worried about negative impacts of the 
system on students’ motivation to stay in 
school, staff morale, and the high school 
curriculum. For instance, a number of 
educators feared that the focus on state 
accountability had narrowed the 
curricula that they could offer, and 
poorly served students whose interests 
were traditionally accommodated by 
high school programs that helped keep 
them interested in school. In addition to 
concerns that vocational, arts, and other 
kinds of coursework were being 
squeezed by the new accountability 
programs, some educators worried that 
tests had narrowed the content of 
English and mathematics courses, 
rigidified instructional routines, and 
reduced the academic experiences for 
low-performing students. A recent 
article about a persistently failing high 
school in California observed that low- 
performing students in 9 th and 10 th grade 
were placed in intensive basic s ki lls 
classes, leaving them little time to 
pursue other academic subjects like 
science or social studies. This strategy, 
prescribed by a state intervention team, 
is similar to that being implemented in 
about two dozen other California high 
schools that have failed to meet 
achievement goals (Munzo, 2004). 

Nevertheless, despite these fears and 
problems, the pressures thrust upon high 
schools by the accountability system 
generated concerted action. Staff in two 
of the Michigan schools that expressed 
the most criticism of testing and 
accountability were, in fact, more active 
than staff in the other schools we studied 
there. Part of the explanation lies in our 
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parallel finding that educators across the 
states had come to accept the 
fundamental premise of standards, and 
saw the aim of common, strong 
academic goals as compatible with good 
educational practice. They also had 
come to accept the idea of performance- 
based accountability. We heard praise 
for common measures to calibrate 
teachers’ expectations, and educators 
agreed that the public should hold 
students and educators to account for 
meeting certain outcomes. In addition, 
many believed that, at least in one form 
or another, these public policy objectives 
were here to stay. In the end, pressure 
and acceptance of the intent of reform 
contributed to a substantial amount of 
response across the six states in our 
study. 

Yet while we found a substantial 
amount of accountability-related action 
across the states, it is also crucial to 
recognize marked variations in both the 
level and the nature of schools’ 
responses within any particular state. 

Level of Response 

While we found a generally higher 
level of response in high-stakes systems, 
consequences were not sufficient in and 
of themselves to motivate action 
consistently across districts or schools. 
Even the lowest performing schools in 
these states sometimes felt little press, 
and reacted only minimally. We also 
encountered both strong and weak 
responses among high schools with 
different performance records in the 
low-stakes accountability systems of 
Pennsylvania and Michigan. Our 
analysis found that an extremely 
important factor in whether or not high 
schools were active was whether they 
were located in a district that took a 



strong stand on accountability, 
mandating or in other ways encouraging 
their schools to act on its behalf. While a 
few schools still resisted these pressures, 
district advocacy was crucial across all 
of our sample states. 

Nature of Response 

Level of response does not reveal 
anything about the nature or quality of 
the actions that high schools selected, or 
whether the efforts were likely to have 
significant effects on teaching and 
learning. While we did not observe 
instruction or attempt to evaluate 
implementation in any way, we analyzed 
whether the adopted initiatives intended 
to change teaching and curriculum, and 
whether they were designed to reach a 
broad or narrow group of students, under 
the assumption that efforts targeted on 
teaching and curriculum for broader 
groups of students would hold greater 
potential for improvement. 

A majority of the accountability- 
related actions undertaken in these high 
schools, in fact, did target changes in 
curriculum and instruction, such as 
aligning the curriculum to state 
standards or adding new basic skills and 
advanced academic courses. Efforts to 
improve students’ ability to read 
appeared across the majority of our 
schools. Of course, reading is 
prerequisite for high school academic 
courses, and reading problems are 
highlighted by state tests, even in fields 
like mathematics. Action in this realm 
ranged from special reading courses or 
remedial reading programs to the 
creation of a reading department in one 
California high school. Another major 
area of activity was remediation efforts 
like tutoring and test preparation. 

Finally, schools adopted many 
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organizational changes, such as block 
scheduling and daily, sustained silent 
reading time. 

However, the reported actions 
ranged from “quick fixes” that were 
marginal to classroom practice and of 
limited impact, to more fundamental 
efforts to improve the core technology of 
what is taught in schools and how for a 
broad group of students. For example, 
before- or after-school tutoring programs 
did not challenge regular instructional 
practice, and typically were not required 
for students. The ubiquitous test 
preparation activities usually did not 
make major changes in curriculum or 
instruction; test prep was often portrayed 
as an insert into regular lessons. 

Teachers typically selected their own 
professional development activities, 
driven by their own particular interests 
or perceptions of need rather than by any 
common, schoolwide goals or vision of 
instruction. Similarly, organizational 
changes like sustained silent reading 
time were peripheral to regular 
instruction. Some schools adopted block 
scheduling in an effort to change 
teachers’ instructional practices, but 
others used it to find time for additional 
services, like counseling. 

These kinds of incremental or 
marginal efforts stand in stark contrast to 
the more comprehensive initiatives we 
found in some high schools. For 
example, one school rewrote the regular 
curriculum for its lowest level 10 th -grade 
English class. Deeper curriculum 
changes could also be seen in some 
efforts to rewrite and align curricula to 
state standards, and to alter the pacing 
and coverage of courses. One school 
adopted an entirely new integrated math 
curriculum for grades 9-12. In the 
organizational realm, one of our schools 
adopted a comprehensive school reform 



model with organizational changes 
intended to complement new 
instructional strategies. Ninth-grade 
academies or teams were introduced in 
some schools in an effort to improve the 
academic and social experiences for this 
segment of the high school population. 
Unfortunately, however, we found that 
these more far-reaching change 
initiatives were rare occurrences; 
incremental and marginal efforts 
dominated how high schools addressed 
the problems they faced. 

So, if accountability design alone is 
not sufficient to drive consistent, 
comprehensive action across high 
schools, we must ask: What is the mix of 
factors that leads them to respond with 
more promising actions? A partial 
answer lies in how schools did or did not 
organize themselves to search for 
improvement ideas. 

Searching for 
Solutions 

We found that the search and 
decision-making process in a majority of 
our sample of high schools was often 
haphazard and left up to individual 
teachers acting on their own initiative — 
in other words, not well organized. This 
finding ran contrary to our expectations; 
the literature on high schools engaged in 
reform suggests that departments play a 
major role in the school improvement 
process (McLaughlin &Talbert, 2001; 
Siskin, 1994; Siskin & Little, 1995). 

This was clearly not the case in most of 
our high schools. While departments 
sometimes played a vital role in the 
social and professional lives of teachers, 
administrators rarely distributed 
decision-making authority to their 
departments, and teachers rarely 
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described their departments as the locus 
of power in their schools. Teachers in 
many schools reported meeting 
infrequently or on an “as needed” basis. 
Rather than providing a forum for 
collective decision making about 
instruction and instructional change, 
most department meetings focused on 
administrative matters and the 
distribution of information about school 
and district policies and professional 
development opportunities. Similarly, 
while many of the schools in our study 
created schoolwide committees to 
develop state-mandated school 
improvement plans or to conduct needs 
analysis and planning as part of the 
accreditation process, these committees 
rarely played a major role in school 
decision-making and improvement 
processes. 

By and large, teachers in these high 
schools decided independently whether 
to make changes in their practice, 
curriculum, and materials. When 
teachers acted on their own, however, 
they tended to meet accountability 
demands with basic test prep strategies. 
In a few cases, groups of teachers, such 
as those who taught Algebra 1 or ninth- 
grade English, worked together to align 
their materials and instruction with state 
standards and assessments and/or with 
each other. But most often, a teacher’s 
actions impacted only her classroom 
and, at times, only individual students. 
These individual decisions, while 
significant and at times constituting the 
major improvement efforts being made 
in a school, did not add up to a 
schoolwide reform effort. 

In addition to the individualistic 
nature of decision making, questions 
arose about whether teachers had the 
capacity to develop an effective response 
to the external demands of 



accountability, and whether schools 
addressed these issues. For example, 
teachers and administrators in 28 of our 
schools identified teacher skill or 
commitment as problem. High school 
teachers’ ability to develop students’ 
reading skills was of particular concern, 
given that many students entered the 
ninth grade with an elementary-level 
reading ability and were expected to 
perform at high levels on state exams. 
High school teachers typically are not 
trained to teach basic reading skills, and 
this may explain why several high 
schools created separate remedial 
reading classes for students. 

But while a majority of our schools 
recognized these problems, only 10 
described any schoolwide or 
departmental effort to address them. For 
instance, in many schools, 
administrators’ management of 
instruction was quite distant. While 
some instructional concerns received a 
lot of attention, such as student course 
failure rates in schools with severe 
dropout problems, veteran teachers said 
that administrators rarely interacted with 
them about their practice. In addition, 
most department chairs felt powerless to 
make decisions about curriculum and 
instruction, or were uncomfortable 
critiquing their peers and trying to exert 
an influence. Classroom teachers did 
share ideas with their colleagues, 
activities that were often facilitated by 
physical proximity in the school building 
or by a common lunch period (rather 
than department meetings). But it was 
extremely rare to find high school 
teachers visiting each other’s classrooms 
or modeling instruction. 

Accountability policies theorize that 
once schools have identified their needs, 
they will organize themselves to search 
for new strategies to improve student 
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performance. Since current 
accountability expectations challenge the 
traditional missions of the 
comprehensive high school and their 
expectations for students (Siskin, 2004), 
we anticipated that staff would seek 
information from outsiders who could 
share new strategies for meetings these 
targets. We found, however, that while 
the press from state accountability 
focused schools on student achievement, 
it did not lead them to look beyond 
schoolhouse doors for information. 
Rather, teachers drew heavily on their 
own experience or the experience of 
their colleagues to solve a problem. 
Schools and teachers faced both external 
and internal barriers to access 
information. Teachers in rural areas did 
not have physical access to professional 
development opportunities outside their 
district. Teachers and principals reported 
that time constraints and tight budgets 
limited their ability to attend conferences 
and workshops. And many teachers and 
administrators did not seem aware that 
they should, or could, look beyond 
themselves or their colleagues for help. 
In some cases, educators did not know 
other ways to search for new 
information. 

Another premise of the new 
accountability, that schools would use 
data to guide change, often did not 
materialize. We found some consistent 
use of data, particularly in North 
Carolina, where the assessments directly 
linked to high school course content and 
results were returned to teachers in a 
very timely fashion. But in other states, 
like Michigan, some teachers had never 
even seen their students’ test results. The 
lack of data use we found in many states 
stemmed in part from the infrequency of 
tests at the high school level — state tests 
were administered once per year, and 



often only once in the high school years. 
The lack of teacher training in the use of 
test data to analyze and address 
weaknesses in individual student 
performance is also a frequently missing 
link in transforming the theory of data 
use in standards-based reform into 
practice. 

In the end, few schools described 
coherent efforts to bring new ideas or 
information on curriculum and 
instruction to their teachers. Those that 
did seemed to have a history of such 
efforts, appearing to confirm earlier 
research that it takes capacity to build 
capacity in high schools (Carnoy, 

Elmore, & Siskin, 2003; Debray, Parson, 
& Avila, 2003; Hatch, 2002). 

External Information 
and Support 

While the vast majority of actions in 
our high schools were based on 
internally generated decision making, 
our study did reveal a few avenues 
through which outside information 
entered into the process. Of course, 
teachers brought information to the table 
from their professional development 
experiences, as did principals who 
attended workshops and conferences. It 
is likely that these experiences informed 
the choices they made during internal 
discussions. 

Some schools did work more 
directly with outside assistance 
providers. For example, a few states like 
California and North Carolina created 
external support structures directed at 
their lowest performing schools. North 
Carolina hired, trained, and assigned 
school support teams composed of 
veteran administrators and subject matter 
specialists to work with low-performing 
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high schools for an academic year. 
California gave struggling high schools 
grants to hire external evaluators to help 
develop and implement the schools’ 
improvement plans, under a program 
known as the Immediate 
Intervention/Underperforming Schools 
Program (II/USP). A few of our study 
schools developed relationships with 
local universities, regional education 
centers, comprehensive school reform 
providers, or other vendors. 

School districts, however, were the 
most prominent as well as influential 
external agents in our study sites. They 
not only stimulated high schools to act 
on behalf of accountability, but also 
guided the kinds of actions schools took. 
Teachers and administrators in more 
than half of our high schools reported 
that districts either suggested or required 
the use of one or more of the 
improvement strategies in place in their 
schools. Some districts mandated the use 
of programs, instructional strategies, or 
curriculum by all, or by low-performing, 
high schools, or they offered unsolicited 
suggestions of what schools could do. 
Other times, schools requested help from 
their central office staff, who drew from 
their knowledge base or sought out new 
ideas to help the schools. 

Active districts tended to be more 
prescriptive, directing high schools to 
adopt specific strategies and monitoring 
the implementation of these practices. 
Active districts also focused their actions 
on the goal of higher student 
performance, generally on the state 
assessment. Central offices developed 
curriculum and pacing guides to align 
school-level instruction to state 
standards. A few districts initiated 
quarterly benchmark examinations to 
provide feedback on student 
performance on the standards. Some 



assigned instructional coaches or content 
area specialists to develop strategies to 
address their schools’ needs. 

However, many districts in our 
study were not proactive on behalf of 
accountability for high schools. For 
some, elementary and middle schools 
took priority, and high schools were left 
to operate quite autonomously. Other 
districts viewed themselves as one of a 
variety of external resources available 
whenever schools requested help. But 
since these districts responded to what 
schools perceived as their own needs, 
district programs addressed a wide range 
of goals, of which student performance 
was only one. Just as with schools, we 
encountered active and more passive 
districts within high and low stakes 
environments. Little academic research 
has explored what motivates and helps 
district organizations intervene on behalf 
of state accountability goals, particularly 
at the high school level. Our study sheds 
some light on this question. 

Active District 
Intervention 

Three factors appear to be related to 
whether districts pressed their high 
schools to adopt actions on behalf of 
accountability. One was the prior 
performance of the high school. Those 
districts that had particularly low- 
performing high schools as measured by 
their state accountability system were 
more likely to take action with those 
schools, especially in states that had 
sanctions for low-performing high 
schools or students. 

However, a second factor interacted 
with school performance to influence 
district response: the size of the district 
central office. (See also, for example, 
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Hannaway & Kimball, 2001.) Did the 
district have sufficient human resources 
to work with its high schools on an 
ongoing basis? For example, in one 
study district with a low-performing 
school, the small size of the central 
office made it difficult for staff to 
provide the school with a high level of 
support. The few staff members were 
consumed with providing schools with 
the basics of school operation — 
personnel, finance, materials, and 
information — and with responding to the 
testing and data requirements of the state 
accountability system. Yet even large 
districts did not always have sufficient 
resources to work with all of their 
schools. In Florida, for example, county 
districts with large staffs but also large 
numbers of high schools targeted their 
assistance to their lowest performing 
ones, those that received state 
accountability grades of D and F, 
because they were subject to state 
sanctions. Middle-performing high 
schools received limited support, 
although they may soon be designated as 
in need of improvement under the No 
Child Left Behind Act (NCLB). 

A third factor, district leadership 
was also critical, particularly the 
existence of a few key administrators 
who established school improvement as 
a goal, identified a set of improvement 
strategies, and saw the district’s role as 
one of directing schools to embrace 
these strategies. Districts with leaders 
that were actively supportive of their 
state accountability system, regardless of 
the strength of that system, the size of 
the district office, or the relative 
performance of their high schools, often 
stimulated more active responses in their 
schools as well. Renaissance City, a 
small district in Pennsylvania, illustrates 
the point. When it was placed on the 



state warning list for poor performance, 
the long-time superintendent there 
decided that his earlier approach of 
delegating school improvement 
initiatives to the schools had failed. He 
started to recentralize control over the 
schools, and held them accountable for 
raising scores on the Pennsylvania 
System of School Assessment (PSSA). 
This led the high school principal to do 
the same and to implement strategies and 
discussions centered around improving 
student achievement on PSSA. 

Similarly, a small district in North 
Carolina set high expectations for its 
schools, used test score data to focus 
teachers and schools on student 
performance, and directed its limited 
resources to areas with the greatest need. 
Another district in that same state 
pushed even its highest performing 
schools to raise the percentage of 
students meeting state standards. 
Superintendents in these small districts 
use their “bully pulpit” to focus attention 
on issues. 

School Resistance 

A high level of district activity was 
not in and of itself sufficient to trigger 
change initiatives in high schools. Some 
study schools grudgingly adopted district 
improvement efforts, while others 
resisted them outright. Some schools at 
the higher ends of the accountability 
spectrum in our sample felt they could 
afford to ignore the press of external 
accountability. For example, Medal 
County, North Carolina, undertook a 
series of actions to improve high school 
instruction. One school was very 
responsive to district initiatives, but staff 
in the second high school felt that since 
they had performed well on state exams, 
they could afford to ignore these efforts. 
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They also thought that they held higher 
goals and standards than the state, and 
had a forceful principal who enabled 
them to operate more independently. At 
the other end of the spectrum, two low- 
performing California schools did little 
to respond to district pressure and 
support in part because of low staff 
morale, a cynical view of state policy 
expectations, and the feeling that 
consequences would never really befall 
their students or themselves. 

Conclusions 

One can draw several conclusions 
from our research in these 48 
underperforming high schools. One is 
that accountability can be a powerful 
force for change in high schools, despite 
the conventional image of high schools 
as recalcitrant organizations. But the 
potential of high schools’ response to 
external accountability depends in part 
on their ability and willingness to bring 
in fresh ideas to the challenges posed by 
policies that ask them to educate all 
students to high levels of academic 
achievement. As one scholar wrote: 

High schools . . . are being asked to 
take on a new task — something they 
were not designed to do — to prepare 
students for a defined minimum 
academic standard, and to get all 
students to graduate by achieving 
that standard. We have certainly not 
organized high schools so that all 
students would take the same content, 
or meet the same standards to 
graduate. In fact, comprehensive high 
schools were historically designed to 
do precisely the opposite; since 
highly influential midcentury Conant 
report, their design imperative has 
been to serve democratic purposes 



and accommodate diverse student 
populations by creating a wide range 
of programs, and a differentiated 
curriculum. (Siskin, 2003, pp. 176— 
177) 

This agenda poses new challenges for 
high schools, and demands new 
solutions that reach outside of teachers’ 
and administrators’ current capacities, 
such as in teaching basic reading skills. 

Further, although research on well- 
functioning high schools shows, as 
indicated previously, that departments 
can play an important role in change 
processes, the high schools in our study, 
performing below expectations, did not 
have strong departments or many other 
formal mechanisms for discussing or 
intervening in instructional practice. 
Finding ways to build organizational 
authority and structures, and/or a culture 
of communication around instructional 
issues, needs priority. Just as landscape 
architects will follow trampled grass by 
public buildings to determine where to 
install sidewalks, developing capacity in 
these kinds of schools may require 
mapping and using more informal 
channels of communication while more 
formal mechanisms are strengthened. 

Districts, certainly, were the most 
important external organization 
influencing these high schools. While in 
recent years questions have arisen about 
the efficacy of district administration, 
our work on this sample of below- 
average high schools concluded that 
parent districts were vital in whether or 
not state policy goals were transmitted 
into school-level action. Moreover, 
districts were the main source of 
guidance and support for high schools, 
far outdistancing third-party providers in 
helping high schools search for 
solutions. While third-party providers 
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are extremely common at the elementary 
and middle school level, we found this 
“supply” of support was surprisingly 
thin across our sample of schools. Who 
can these schools turn to for new ideas 
and support to handle the challenges 
they face? 

While one reasonable solution may 
be expanding the numbers of third-party 
providers, districts are already 
ubiquitous, and have significant 
institutional power and authority over 
schools. Building district capacity, and 
finding effective incentives for their 
intervention in high schools, is critical to 
school improvement. In addition to its 
focus on schools, NCLB holds districts 
accountable for the performance of their 
students and responsible for assisting 
schools that do not meet state standards. 
Districts must provide ongoing technical 
assistance as schools develop and 
implement school improvement plans. 
They are to help schools analyze student 
achievement data, implement 
professional development, and put in 
place a new curriculum or instructional 
practices that have shown evidence of 
effectiveness. Districts that do not 
themselves meet performance 
expectations can be subject to state 
intervention. 

But will these mandates and 
incentives be sufficient to stimulate 
more, and more effective, district 
guidance to high schools? Our research 
here suggests, at least, that stakes alone 
will be insufficient to spur consistent 
action across districts (or schools, for 
that matter); activism is more than a 
matter of mandate — it is deeply 
intertwined with the capabilities of 
people and their organizations to 
respond, their knowledge, their 
resources, and their motivation. 
Motivation to respond is influenced in 



part by how districts and schools 
perceive where they “fit” in the 
accountability ratings and by whether 
accountability is viewed as too 
demanding or sufficiently demanding or 
not demanding enough. Response is also 
influenced by capacity. Districts need 
strong and stable leadership, 
accompanied by staff who are 
knowledgeable about high schools and 
improvement strategies in those 
particular institutions. Leadership 
turnover is one problem, particularly in 
big cities. For instance, Michigan 
recently reported that in 2003-2004, the 
rate of retirement of superintendents was 
double that of the previous year. Fewer 
people are applying for these positions, 
citing in part the pressures under NCLB 
(MacDonald, 2004). Accountability 
policies often ignore or give minimal 
attention to these district issues, and 
NCLB is no exception. States must 
attend to the capacity of school districts, 
just as they expect districts to attend to 
the capacity of their low-performing 
schools. 
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